Hadoop Common
  1. Hadoop Common
  2. HADOOP-3678

Avoid spurious "DataXceiver: java.io.IOException: Connection reset by peer" errors in DataNode log

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.17.0
    • Fix Version/s: 0.17.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Avoid spurious exceptions logged at DataNode when clients read from DFS.

      Description

      When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.

      1. HADOOP-3678-branch-17.patch
        1 kB
        Raghu Angadi
      2. HADOOP-3678.patch
        3 kB
        Raghu Angadi
      3. HADOOP-3678.patch
        3 kB
        Raghu Angadi

        Issue Links

          Activity

          Raghu Angadi made changes -
          Affects Version/s 0.17.0 [ 12312913 ]
          Affects Version/s 0.18.2 [ 12313424 ]
          Raghu Angadi made changes -
          Link This issue is related to HADOOP-4862 [ HADOOP-4862 ]
          Cosmin Lehene made changes -
          Affects Version/s 0.18.2 [ 12313424 ]
          Affects Version/s 0.17.0 [ 12312913 ]
          Owen O'Malley made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Owen O'Malley made changes -
          Fix Version/s 0.18.0 [ 12312972 ]
          Raghu Angadi made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Raghu Angadi made changes -
          Priority Major [ 3 ] Blocker [ 1 ]
          Fix Version/s 0.17.2 [ 12313296 ]
          Raghu Angadi made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Raghu Angadi made changes -
          Attachment HADOOP-3678-branch-17.patch [ 12385238 ]
          Raghu Angadi made changes -
          Resolution Fixed [ 1 ]
          Release Note Avoid spurious exceptions logged at DataNode when clients read from DFS.
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Tsz Wo Nicholas Sze made changes -
          Hadoop Flags [Reviewed]
          Raghu Angadi made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Raghu Angadi made changes -
          Attachment HADOOP-3678.patch [ 12385136 ]
          Raghu Angadi made changes -
          Attachment HADOOP-3678.patch [ 12385070 ]
          Raghu Angadi made changes -
          Fix Version/s 0.18.0 [ 12312972 ]
          Raghu Angadi made changes -
          Field Original Value New Value
          Description
          When a client reads data using read(), it closes the sockets after it is done. Often it might read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed of end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
          When a client reads data using read(), it closes the sockets after it is done. Often it might not read till the end of a block. The datanode on the other side keeps writing data until the client connection is closed or end of the block is reached. If the client does not read till the end of the block, Datanode writes an error message and stack trace to the datanode log. It should not. This is not an error and it just pollutes the log and confuses the user.
          Raghu Angadi created issue -

            People

            • Assignee:
              Raghu Angadi
              Reporter:
              Raghu Angadi
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development