Hadoop Common
  1. Hadoop Common
  2. HADOOP-128

Failure to replicate dfs block kills client

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.1.1
    • Fix Version/s: 0.2.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      ~200 node linux cluster (kernel 2.6, redhat, 2 hyper threaded cpus)

      Description

      When the datanode gets an exception, which is logged as:

      060407 155835 13 DataXCeiver
      java.io.EOFException
      at java.io.DataInputStream.readFully(DataInputStream.java:178)
      at java.io.DataInputStream.readLong(DataInputStream.java:380)
      at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:462)
      at java.lang.Thread.run(Thread.java:595)

      It closes the user's connection to the data node, which causes the client to get an IOException from:

      at java.io.DataInputStream.readFully(DataInputStream.java:178)
      at java.io.DataInputStream.readLong(DataInputStream.java:380)
      at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.internalClose(DFSClient.java:883)

      1. conf.patch
        0.6 kB
        Owen O'Malley
      2. datanode.no-ws-diff
        10 kB
        Owen O'Malley
      3. datanode-mirroring.patch
        31 kB
        Owen O'Malley

        Activity

        Owen O'Malley created issue -
        Owen O'Malley made changes -
        Field Original Value New Value
        Attachment datanode-mirroring.patch [ 12325303 ]
        Owen O'Malley made changes -
        Attachment datanode.no-ws-diff [ 12325305 ]
        Owen O'Malley made changes -
        Attachment conf.patch [ 12325306 ]
        Doug Cutting made changes -
        Fix Version/s 0.2 [ 12310813 ]
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Doug Cutting made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Doug Cutting made changes -
        Workflow jira [ 12354988 ] no reopen closed [ 12373069 ]
        Doug Cutting made changes -
        Workflow no reopen closed [ 12373069 ] no-reopen-closed [ 12373405 ]
        Doug Cutting made changes -
        Workflow no-reopen-closed [ 12373405 ] no-reopen-closed, patch-avail [ 12377715 ]
        Owen O'Malley made changes -
        Component/s dfs [ 12310710 ]

          People

          • Assignee:
            Owen O'Malley
            Reporter:
            Owen O'Malley
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development