Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.0.0-alpha
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      While running the tests, I have seen this exceptions.Tests passed.
      Not sure this is a problem.

      2012-04-26 23:15:51,763 WARN hdfs.DFSClient (DFSOutputStream.java:run(710)) - DFSOutputStream ResponseProcessor exception for block BP-1372255573-49.249.124.17-1335462329685:blk_-8435040801555580201_1005
      java.io.EOFException: Premature EOF: no length prefix available
      at org.apache.hadoop.hdfs.protocol.HdfsProtoUtil.vintPrefixed(HdfsProtoUtil.java:162)
      at org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.readFields(PipelineAck.java:95)
      at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer$ResponseProcessor.run(DFSOutputStream.java:657)
      Exception in thread "DataXceiver for client /127.0.0.1:52323 [Cleaning up]" java.lang.NullPointerException
      at org.apache.hadoop.ipc.Server$Listener.getAddress(Server.java:669)
      at org.apache.hadoop.ipc.Server.getListenerAddress(Server.java:1988)
      at org.apache.hadoop.hdfs.server.datanode.DataNode.getIpcPort(DataNode.java:882)
      at org.apache.hadoop.hdfs.server.datanode.DataNode.getDisplayName(DataNode.java:863)
      at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:171)
      at java.lang.Thread.run(Unknown Source)

      1. hdfs-3328.txt
        1 kB
        Eli Collins

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1040 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1040/)
        HDFS-3328. NPE in DataNode.getIpcPort. Contributed by Eli Collins (Revision 1336480)

        Result = FAILURE
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336480
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1040 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1040/ ) HDFS-3328 . NPE in DataNode.getIpcPort. Contributed by Eli Collins (Revision 1336480) Result = FAILURE eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336480 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1075 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1075/)
        HDFS-3328. NPE in DataNode.getIpcPort. Contributed by Eli Collins (Revision 1336480)

        Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336480
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1075 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1075/ ) HDFS-3328 . NPE in DataNode.getIpcPort. Contributed by Eli Collins (Revision 1336480) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336480 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2234 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2234/)
        HDFS-3328. NPE in DataNode.getIpcPort. Contributed by Eli Collins (Revision 1336480)

        Result = ABORTED
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336480
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2234 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2234/ ) HDFS-3328 . NPE in DataNode.getIpcPort. Contributed by Eli Collins (Revision 1336480) Result = ABORTED eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336480 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Hide
        Eli Collins added a comment -

        Thanks guys. I've committed this to trunk and merged to branch-2 and branch-2-alpha.

        Show
        Eli Collins added a comment - Thanks guys. I've committed this to trunk and merged to branch-2 and branch-2-alpha.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2217 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2217/)
        HDFS-3328. NPE in DataNode.getIpcPort. Contributed by Eli Collins (Revision 1336480)

        Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336480
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2217 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2217/ ) HDFS-3328 . NPE in DataNode.getIpcPort. Contributed by Eli Collins (Revision 1336480) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336480 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2292 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2292/)
        HDFS-3328. NPE in DataNode.getIpcPort. Contributed by Eli Collins (Revision 1336480)

        Result = SUCCESS
        eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336480
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2292 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2292/ ) HDFS-3328 . NPE in DataNode.getIpcPort. Contributed by Eli Collins (Revision 1336480) Result = SUCCESS eli : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1336480 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
        Hide
        Aaron T. Myers added a comment -

        +1, the patch looks good to me.

        Show
        Aaron T. Myers added a comment - +1, the patch looks good to me.
        Hide
        Daryn Sharp added a comment -

        +1 Ok, thanks for the clarification!

        Show
        Daryn Sharp added a comment - +1 Ok, thanks for the clarification!
        Hide
        Eli Collins added a comment -

        Thanks Daryn. This puts the behavior back as it was previously - metrics and toString previously used the xfer port. It would only matter if people care about the particular port value in the "DataNodeActivity-<hostname>-<port>" key. I suspect most people care about the hostname, the port is used to make it unique for the case of running multiple DNs on the same host.

        Show
        Eli Collins added a comment - Thanks Daryn. This puts the behavior back as it was previously - metrics and toString previously used the xfer port. It would only matter if people care about the particular port value in the "DataNodeActivity-<hostname>-<port>" key. I suspect most people care about the hostname, the port is used to make it unique for the case of running multiple DNs on the same host.
        Hide
        Daryn Sharp added a comment -

        Nice, this looks like the more appropriate port for toString(). However, the metrics are also inited with the display name. Does it matter for metrics collection as to whether the xfer or ipc port is used?

        Show
        Daryn Sharp added a comment - Nice, this looks like the more appropriate port for toString() . However, the metrics are also inited with the display name. Does it matter for metrics collection as to whether the xfer or ipc port is used?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12525954/hdfs-3328.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2387//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2387//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12525954/hdfs-3328.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2387//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2387//console This message is automatically generated.
        Hide
        Eli Collins added a comment -

        Patch attached. Use the xfer port rather than the IPC port in DataNode#getDisplayName. This was the behavior before HDFS-3171, and doesn't NPE because the xfer port is set statically by the tests rather than waiting for an IPC server to come up. Also, let's remove @VisibleForTesting from getIpcPort since it's used in non-test code paths.

        Show
        Eli Collins added a comment - Patch attached. Use the xfer port rather than the IPC port in DataNode#getDisplayName. This was the behavior before HDFS-3171 , and doesn't NPE because the xfer port is set statically by the tests rather than waiting for an IPC server to come up. Also, let's remove @VisibleForTesting from getIpcPort since it's used in non-test code paths.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I think it may cause TestHFlush to fail. Anyway, we definitely should fix the NPE.

        Show
        Tsz Wo Nicholas Sze added a comment - I think it may cause TestHFlush to fail. Anyway, we definitely should fix the NPE.

          People

          • Assignee:
            Eli Collins
            Reporter:
            Uma Maheswara Rao G
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development