Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-770

SocketTimeoutException: timeout while waiting for channel to be ready for read

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.20.1
    • Fix Version/s: None
    • Labels:
      None
    • Environment:

      Ubuntu Linux 8.04

      Description

      We're having issues with timeouts occurring in our client: for some reason, a timeout of 63000 milliseconds is triggered while writing HDFS data. Since we currently have a single-server setup, this results in our client terminating with a "All datanodes are bad" IOException.

      We're running all services, including the client, on our single server, so it cannot be a network error. The load on the client is extremely low during this period: only a few kilobytes a minute were being written around the time the error occured.

      After browsing a bit online, a lot of people talk about setting "dfs.datanode.socket.write.timeout" to 0 as a solution for this problem. Due to the low load of our system during this period, however, I do feel this is a real error and a timeout that should not be occurring. I have attached 3 logs of the namenode, datanode and client.

      It could be that this is related to http://issues.apache.org/jira/browse/HDFS-693

      Any pointers on how I can assist to resolve this issue will be greatly appreciated.

      1. namenode.txt
        3 kB
        Leon Mergen
      2. filewriter.cpp
        0.6 kB
        Zheng Shao
      3. datanode.txt
        4 kB
        Leon Mergen
      4. client.txt
        3 kB
        Leon Mergen

        Issue Links

          Activity

          Leon Mergen created issue -
          Leon Mergen made changes -
          Field Original Value New Value
          Attachment datanode.txt [ 12424822 ]
          Attachment namenode.txt [ 12424823 ]
          Attachment client.txt [ 12424824 ]
          Zheng Shao made changes -
          Attachment filewriter.cpp [ 12428659 ]
          Todd Lipcon made changes -
          Link This issue relates to HDFS-1075 [ HDFS-1075 ]
          Todd Lipcon made changes -
          Link This issue is related to HDFS-561 [ HDFS-561 ]
          Eli Collins made changes -
          Link This issue is related to HDFS-693 [ HDFS-693 ]
          Todd Lipcon made changes -
          Link This issue relates to HADOOP-4584 [ HADOOP-4584 ]

            People

            • Assignee:
              Unassigned
              Reporter:
              Leon Mergen
            • Votes:
              3 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:

                Development