Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-3179

Improve the error message: DataStreamer throw an exception, "nodes.length != original.length + 1" on single datanode cluster

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.2
    • Fix Version/s: 2.0.0-alpha
    • Component/s: hdfs-client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Create a single datanode cluster

      disable permissions
      enable webhfds
      start hdfs
      run the test script

      expected result:
      a file named "test" is created and the content is "testtest"

      the result I got:
      hdfs throw an exception on the second append operation.

      ./test.sh 
      {"RemoteException":{"exception":"IOException","javaClassName":"java.io.IOException","message":"Failed to add a datanode: nodes.length != original.length + 1, nodes=[127.0.0.1:50010], original=[127.0.0.1:50010]"}}
      

      Log in datanode:

      2012-04-02 14:34:21,058 WARN org.apache.hadoop.hdfs.DFSClient: DataStreamer Exception
      java.io.IOException: Failed to add a datanode: nodes.length != original.length + 1, nodes=[127.0.0.1:50010], original=[127.0.0.1:50010]
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.findNewDatanode(DFSOutputStream.java:778)
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.addDatanode2ExistingPipeline(DFSOutputStream.java:834)
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:930)
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:461)
      2012-04-02 14:34:21,059 ERROR org.apache.hadoop.hdfs.DFSClient: Failed to close file /test
      java.io.IOException: Failed to add a datanode: nodes.length != original.length + 1, nodes=[127.0.0.1:50010], original=[127.0.0.1:50010]
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.findNewDatanode(DFSOutputStream.java:778)
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.addDatanode2ExistingPipeline(DFSOutputStream.java:834)
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:930)
      	at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:461)
      

      test.sh

      #!/bin/sh
      
      echo "test" > test.txt
      
      curl -L -X PUT "http://localhost:50070/webhdfs/v1/test?op=CREATE"
      
      curl -L -X POST -T test.txt "http://localhost:50070/webhdfs/v1/test?op=APPEND"
      curl -L -X POST -T test.txt "http://localhost:50070/webhdfs/v1/test?op=APPEND"
      
      
      1. h3179_20120403.patch
        4 kB
        Tsz Wo Nicholas Sze

        Activity

        Zhanwei.Wang created issue -
        Tsz Wo Nicholas Sze made changes -
        Field Original Value New Value
        Attachment h3179_20120403.patch [ 12521272 ]
        Tsz Wo Nicholas Sze made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Assignee Tsz Wo (Nicholas), SZE [ szetszwo ]
        Tsz Wo Nicholas Sze made changes -
        Summary failed to append data, DataStreamer throw an exception, "nodes.length != original.length + 1" on single datanode cluster Improve the error message: DataStreamer throw an exception, "nodes.length != original.length + 1" on single datanode cluster
        Issue Type Bug [ 1 ] Improvement [ 4 ]
        Priority Critical [ 2 ] Major [ 3 ]
        Component/s hdfs client [ 12312928 ]
        Component/s data-node [ 12312927 ]
        Tsz Wo Nicholas Sze made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 2.0.0 [ 12320353 ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Tsz Wo Nicholas Sze
            Reporter:
            Zhanwei.Wang
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development