Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3555

idle client socket triggers DN ERROR log (should be INFO or DEBUG)

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.2
    • Fix Version/s: 2.0.2-alpha
    • Component/s: datanode, hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Datanode service is logging java.net.SocketTimeoutException at ERROR level.
      This message indicates that the datanode is not able to send data to the client because the client has stopped reading. This message is not really a cause for alarm and should be INFO level.

      2012-06-18 17:47:13 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode DatanodeRegistration(x.x.x.x:50010, storageID=DS-196671195-10.10.120.67-50010-1334328338972, infoPort=50075, ipcPort=50020):DataXceiver
      java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/10.10.120.67:50010 remote=/10.10.120.67:59282]
      at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246)
      at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:159)
      at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:198)
      at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendChunks(BlockSender.java:397)
      at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:493)
      at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:267)
      at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:163)

      1. hdfs-3555-3.txt
        3 kB
        Andy Isaacson
      2. hdfs-3555-2.txt
        2 kB
        Andy Isaacson
      3. hdfs-3555.patch
        3 kB
        Andy Isaacson

        Activity

        Hide
        Andy Isaacson added a comment -

        Simple to reproduce, just do

            FileSystem fs = FileSystem.get(new Configuration());
            DataInputStream f = fs.open(new Path(args[0]));
            f.read(new byte[1024]);
            Thread.sleep(500 * 1000);
        

        The DN eventually gives up on the client socket and ERRORs the DataXceiver SocketTimeoutException. This is definitely not ERROR worthy, I would say DEBUG or INFO at most.

        Show
        Andy Isaacson added a comment - Simple to reproduce, just do FileSystem fs = FileSystem.get( new Configuration()); DataInputStream f = fs.open( new Path(args[0])); f.read( new byte [1024]); Thread .sleep(500 * 1000); The DN eventually gives up on the client socket and ERRORs the DataXceiver SocketTimeoutException. This is definitely not ERROR worthy, I would say DEBUG or INFO at most.
        Hide
        Jeff Lord added a comment -

        Does this look right?


        .../hadoop/hdfs/server/datanode/DataXceiver.java | 2 +-
        1 files changed, 1 insertions, 1 deletions

        diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
        index 6c280d8..2fb5878 100644
        — a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
        +++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java
        @@ -241,7 +241,7 @@ class DataXceiver extends Receiver implements Runnable {
        } catch(IOException e)

        { String msg = "opReadBlock " + block + " received exception " + e; LOG.info(msg); - sendResponse(s, ERROR, msg, dnConf.socketWriteTimeout); + sendResponse(s, INFO, msg, dnConf.socketWriteTimeout); throw e; }

        Show
        Jeff Lord added a comment - Does this look right? — .../hadoop/hdfs/server/datanode/DataXceiver.java | 2 +- 1 files changed, 1 insertions , 1 deletions diff --git a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java index 6c280d8..2fb5878 100644 — a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java +++ b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiver.java @@ -241,7 +241,7 @@ class DataXceiver extends Receiver implements Runnable { } catch(IOException e) { String msg = "opReadBlock " + block + " received exception " + e; LOG.info(msg); - sendResponse(s, ERROR, msg, dnConf.socketWriteTimeout); + sendResponse(s, INFO, msg, dnConf.socketWriteTimeout); throw e; } –
        Hide
        Andy Isaacson added a comment -

        The error is a little different on 2.0:

        2012-06-21 18:28:36,251 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: BlockSender.sendChunks() exception: 
        java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/192.168.122.87:50010 remote=
        /192.168.122.3:51436]
                at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246)
                at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:164)
                at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:203)
                at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendPacket(BlockSender.java:482)
                at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:634)
                at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:252)
                at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:88)
                at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:63)
                at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:189)
                at java.lang.Thread.run(Thread.java:679)
        2012-06-21 18:28:36,252 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /192.168.122.87:50010, dest: /192.168.122.3:51436, bytes: 53697024, op: HDFS_READ, cliID: D
        FSClient_NONMAPREDUCE_-1988072026_1, offset: 0, srvID: DS-706541979-127.0.1.1-50010-1339724203679, blockid: BP-882164591-127.0.1.1-1339723952222:blk_-1935427635464392086_1010, duration: 
        482450603444
        2012-06-21 18:28:36,252 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(192.168.122.87, storageID=DS-706541979-127.0.1.1-50010-1339724203679, infoPort=50075, i
        pcPort=50020, storageInfo=lv=-40;cid=CID-02666c8e-a05e-480f-94df-f5226414f260;nsid=1569472409;c=0):Got exception while serving BP-882164591-127.0.1.1-1339723952222:blk_-19354276354643920
        86_1010 to /192.168.122.3:51436
        java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/192.168.122.87:50010 remote=
        /192.168.122.3:51436]
                at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246)
                at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:164)
                at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:203)
                at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendPacket(BlockSender.java:482)
                at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:634)
                at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:252)
                at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:88)
                at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:63)
                at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:189)
                at java.lang.Thread.run(Thread.java:679)
        2012-06-21 18:28:36,253 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: ubu-cdh-3:50010:DataXceiver error processing READ_BLOCK operation  src: /192.168.122.3:51436 dest: /192.168.122.87:50010
        java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/192.168.122.87:50010 remote=/192.168.122.3:51436]
                at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246)
                at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:164)
                at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:203)
                at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendPacket(BlockSender.java:482)
                at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:634)
                at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:252)
                at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:88)
                at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:63)
                at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:189)
                at java.lang.Thread.run(Thread.java:679)
        
        Show
        Andy Isaacson added a comment - The error is a little different on 2.0: 2012-06-21 18:28:36,251 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: BlockSender.sendChunks() exception: java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/192.168.122.87:50010 remote= /192.168.122.3:51436] at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246) at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:164) at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:203) at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendPacket(BlockSender.java:482) at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:634) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:252) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:88) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:63) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:189) at java.lang. Thread .run( Thread .java:679) 2012-06-21 18:28:36,252 INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: /192.168.122.87:50010, dest: /192.168.122.3:51436, bytes: 53697024, op: HDFS_READ, cliID: D FSClient_NONMAPREDUCE_-1988072026_1, offset: 0, srvID: DS-706541979-127.0.1.1-50010-1339724203679, blockid: BP-882164591-127.0.1.1-1339723952222:blk_-1935427635464392086_1010, duration: 482450603444 2012-06-21 18:28:36,252 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(192.168.122.87, storageID=DS-706541979-127.0.1.1-50010-1339724203679, infoPort=50075, i pcPort=50020, storageInfo=lv=-40;cid=CID-02666c8e-a05e-480f-94df-f5226414f260;nsid=1569472409;c=0):Got exception while serving BP-882164591-127.0.1.1-1339723952222:blk_-19354276354643920 86_1010 to /192.168.122.3:51436 java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/192.168.122.87:50010 remote= /192.168.122.3:51436] at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246) at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:164) at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:203) at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendPacket(BlockSender.java:482) at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:634) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:252) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:88) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:63) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:189) at java.lang. Thread .run( Thread .java:679) 2012-06-21 18:28:36,253 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: ubu-cdh-3:50010:DataXceiver error processing READ_BLOCK operation src: /192.168.122.3:51436 dest: /192.168.122.87:50010 java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/192.168.122.87:50010 remote=/192.168.122.3:51436] at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246) at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:164) at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:203) at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendPacket(BlockSender.java:482) at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:634) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:252) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:88) at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:63) at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:189) at java.lang. Thread .run( Thread .java:679)
        Hide
        Andy Isaacson added a comment -

        Super not obvious, the ERROR is coming from hdfs/server/datanode/BlockSender.java following horrifyingness:

        489     } catch (IOException e) {
        490       /* Exception while writing to the client. Connection closure from
        491        * the other end is mostly the case and we do not care much about
        492        * it. But other things can go wrong, especially in transferTo(),
        493        * which we do not want to ignore.
        494        *
        495        * The message parsing below should not be considered as a good
        496        * coding example. NEVER do it to drive a program logic. NEVER.
        497        * It was done here because the NIO throws an IOException for EPIPE.
        498        */
        499       String ioem = e.getMessage();
        500       if (!ioem.startsWith("Broken pipe") && !ioem.startsWith("Connection reset")) {
        501         LOG.error("BlockSender.sendChunks() exception: ", e);
        502       }
        503       throw ioeToSocketException(e);
        504     }
        
        Show
        Andy Isaacson added a comment - Super not obvious, the ERROR is coming from hdfs/server/datanode/BlockSender.java following horrifyingness: 489 } catch (IOException e) { 490 /* Exception while writing to the client. Connection closure from 491 * the other end is mostly the case and we do not care much about 492 * it. But other things can go wrong, especially in transferTo(), 493 * which we do not want to ignore. 494 * 495 * The message parsing below should not be considered as a good 496 * coding example. NEVER do it to drive a program logic. NEVER. 497 * It was done here because the NIO throws an IOException for EPIPE. 498 */ 499 String ioem = e.getMessage(); 500 if (!ioem.startsWith( "Broken pipe" ) && !ioem.startsWith( "Connection reset" )) { 501 LOG.error( "BlockSender.sendChunks() exception: " , e); 502 } 503 throw ioeToSocketException(e); 504 }
        Hide
        Andy Isaacson added a comment -

        Here's a proposed fix to downgrade the timeout exception from LOG.error to LOG.debug. Compiles, but haven't tested yet as my test cluster is busy.

        Show
        Andy Isaacson added a comment - Here's a proposed fix to downgrade the timeout exception from LOG.error to LOG.debug. Compiles, but haven't tested yet as my test cluster is busy.
        Hide
        Harsh J added a comment -

        I think it can be INFO/WARN, but not ERROR (it isn't that very severe) nor DEBUG (hard to see if clients are misbehaving in this case). Thoughts Andy? Should we really make it DEBUG?

        Also, can we check and lower a write-side ERROR message if one exists for a similar case (reading data from client timing out)?

        Show
        Harsh J added a comment - I think it can be INFO/WARN, but not ERROR (it isn't that very severe) nor DEBUG (hard to see if clients are misbehaving in this case). Thoughts Andy? Should we really make it DEBUG? Also, can we check and lower a write-side ERROR message if one exists for a similar case (reading data from client timing out)?
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12532986/hdfs-3555.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 javadoc. The javadoc tool appears to have generated 13 warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

        org.apache.hadoop.hdfs.TestHFlush

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2684//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2684//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12532986/hdfs-3555.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 javadoc. The javadoc tool appears to have generated 13 warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.TestHFlush +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2684//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2684//console This message is automatically generated.
        Hide
        Andy Isaacson added a comment -

        hard to see if clients are misbehaving in this case.

        I really don't think this is much of a misbehavior, the client just closed the socket after a delay. So I don't think we should log about it by default, even at a low priority. It's bad to cause IOPS and log file growth on the DN when the cluster is operating as designed.

        Since INFO is logged by default, it shouldn't be INFO, but DEBUG I guess.

        Or maybe this should be INFO and I should be pushing to change the default log to WARN.

        Show
        Andy Isaacson added a comment - hard to see if clients are misbehaving in this case. I really don't think this is much of a misbehavior, the client just closed the socket after a delay. So I don't think we should log about it by default, even at a low priority. It's bad to cause IOPS and log file growth on the DN when the cluster is operating as designed. Since INFO is logged by default, it shouldn't be INFO, but DEBUG I guess. Or maybe this should be INFO and I should be pushing to change the default log to WARN.
        Hide
        Andy Isaacson added a comment -

        I went ahead and moved the log message up to INFO. New patch uploaded.

        Show
        Andy Isaacson added a comment - I went ahead and moved the log message up to INFO. New patch uploaded.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12533151/hdfs-3555-2.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2689//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12533151/hdfs-3555-2.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2689//console This message is automatically generated.
        Hide
        Harsh J added a comment -

        Andy,

        This looks okay to go, but can you rebase it please? It does not apply to current trunk.

        Also, is instanceof better than using a specific catch clause for SocketTimeoutException? I've seen clusters where timeouts are logged very frequently under load, and we don't wish to slow the DN down just for logging's sake, if instanceof is expensive to do.

        Thanks!

        Show
        Harsh J added a comment - Andy, This looks okay to go, but can you rebase it please? It does not apply to current trunk. Also, is instanceof better than using a specific catch clause for SocketTimeoutException? I've seen clusters where timeouts are logged very frequently under load, and we don't wish to slow the DN down just for logging's sake, if instanceof is expensive to do. Thanks!
        Hide
        Harsh J added a comment -

        NVM my concern. I got it answered via http://stackoverflow.com/questions/103564/the-performance-impact-of-using-instanceof-in-java

        Please do send in a properly applying patch. +1 as is.

        Show
        Harsh J added a comment - NVM my concern. I got it answered via http://stackoverflow.com/questions/103564/the-performance-impact-of-using-instanceof-in-java Please do send in a properly applying patch. +1 as is.
        Hide
        Andy Isaacson added a comment -

        This looks okay to go, but can you rebase it please? It does not apply to current trunk.

        My mistake, I uploaded "git show -b" which is nice to read but of course doesn't get the indentation correct.

        Also, is instanceof better than using a specific catch clause for SocketTimeoutException?

        If there are two catch clauses, the common code gets duplicated. Currently that's just one line but it's just begging for someone to mistakenly add code to just one of the catch blocks ...
        Given that this codepath is already pretty expensive (we're about to tear down a TCP socket, we've already constructed the Exception) the small additional overhead of instanceof is negligible.

        Show
        Andy Isaacson added a comment - This looks okay to go, but can you rebase it please? It does not apply to current trunk. My mistake, I uploaded "git show -b" which is nice to read but of course doesn't get the indentation correct. Also, is instanceof better than using a specific catch clause for SocketTimeoutException? If there are two catch clauses, the common code gets duplicated. Currently that's just one line but it's just begging for someone to mistakenly add code to just one of the catch blocks ... Given that this codepath is already pretty expensive (we're about to tear down a TCP socket, we've already constructed the Exception) the small additional overhead of instanceof is negligible.
        Hide
        Andy Isaacson added a comment -

        Attaching correctly formatted patch.

        Show
        Andy Isaacson added a comment - Attaching correctly formatted patch.
        Hide
        Harsh J added a comment -

        Thanks Andy. Will commit it in pending jenkins' result.

        Show
        Harsh J added a comment - Thanks Andy. Will commit it in pending jenkins' result.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12535733/hdfs-3555-3.txt
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2769//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12535733/hdfs-3555-3.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2769//console This message is automatically generated.
        Hide
        Harsh J added a comment -

        The hudson buildbot seems to be acting strange lately. I verified that this works manually after applying locally, and hence committing it in now.

        Show
        Harsh J added a comment - The hudson buildbot seems to be acting strange lately. I verified that this works manually after applying locally, and hence committing it in now.
        Hide
        Harsh J added a comment -

        Committed to branch-2 and trunk. Thank you for your many contributions Andy!

        Show
        Harsh J added a comment - Committed to branch-2 and trunk. Thank you for your many contributions Andy!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2505 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2505/)
        HDFS-3555. idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh) (Revision 1359619)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359619
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2505 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2505/ ) HDFS-3555 . idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh) (Revision 1359619) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359619 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2437 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2437/)
        HDFS-3555. idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh) (Revision 1359619)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359619
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2437 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2437/ ) HDFS-3555 . idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh) (Revision 1359619) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359619 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2455 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2455/)
        HDFS-3555. idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh) (Revision 1359619)

        Result = FAILURE
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359619
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2455 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2455/ ) HDFS-3555 . idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh) (Revision 1359619) Result = FAILURE harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359619 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1132 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1132/)
        HDFS-3555. idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh) (Revision 1359619)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359619
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1132 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1132/ ) HDFS-3555 . idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh) (Revision 1359619) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359619 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2508 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2508/)
        HDFS-3555. idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh) (Revision 1359619)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359619
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2508 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2508/ ) HDFS-3555 . idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh) (Revision 1359619) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359619 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2441 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2441/)
        HDFS-3555. idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh) (Revision 1359619)

        Result = SUCCESS
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359619
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2441 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2441/ ) HDFS-3555 . idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh) (Revision 1359619) Result = SUCCESS harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359619 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1100 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1100/)
        HDFS-3555. idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh) (Revision 1359619)

        Result = FAILURE
        harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359619
        Files :

        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1100 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1100/ ) HDFS-3555 . idle client socket triggers DN ERROR log (should be INFO or DEBUG). Contributed by Andy Isaacson. (harsh) (Revision 1359619) Result = FAILURE harsh : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1359619 Files : /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockSender.java

          People

          • Assignee:
            Andy Isaacson
            Reporter:
            Jeff Lord
          • Votes:
            1 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development