Details
-
Bug
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
Description
When running HDFS with docker-compose up hdfs-data-node-1 I get the following error:
[==========] Running 2 tests from 2 test cases. [----------] Global test environment set-up. [----------] 1 test from TestHadoopFileSystem/0, where TypeParam = arrow::io::JNIDriver [ RUN ] TestHadoopFileSystem/0.LargeFile [ OK ] TestHadoopFileSystem/0.LargeFile (1101 ms) [----------] 1 test from TestHadoopFileSystem/0 (1101 ms total) [----------] 1 test from TestHadoopFileSystem/1, where TypeParam = arrow::io::PivotalDriver [ RUN ] TestHadoopFileSystem/1.LargeFile Loading libhdfs3 failed, skipping tests gracefully. IOError: ../src/arrow/io/hdfs-internal.cc:566 code: try_dlopen(libhdfs3_potential_paths, "libhdfs3", shim->handle) Unable to load libhdfs3 Driver not loaded, skipping [ OK ] TestHadoopFileSystem/1.LargeFile (0 ms) [----------] 1 test from TestHadoopFileSystem/1 (0 ms total) [----------] Global test environment tear-down [==========] 2 tests from 2 test cases ran. (1101 ms total) [ PASSED ] 2 tests. (arrow-dev) 21:06 ~/code/arrow/cpp/build (ARROW-2919)$ ./debug/arrow-io-hdfs-test Running main() from gtest_main.cc [==========] Running 24 tests from 2 test cases. [----------] Global test environment set-up. [----------] 12 tests from TestHadoopFileSystem/0, where TypeParam = arrow::io::JNIDriver [ RUN ] TestHadoopFileSystem/0.ConnectsAgain [ OK ] TestHadoopFileSystem/0.ConnectsAgain (885 ms) [ RUN ] TestHadoopFileSystem/0.MultipleClients [ OK ] TestHadoopFileSystem/0.MultipleClients (194 ms) [ RUN ] TestHadoopFileSystem/0.MakeDirectory [ OK ] TestHadoopFileSystem/0.MakeDirectory (139 ms) [ RUN ] TestHadoopFileSystem/0.GetCapacityUsed [ OK ] TestHadoopFileSystem/0.GetCapacityUsed (119 ms) [ RUN ] TestHadoopFileSystem/0.GetPathInfo [ OK ] TestHadoopFileSystem/0.GetPathInfo (229 ms) [ RUN ] TestHadoopFileSystem/0.AppendToFile 18/12/17 21:06:38 WARN hdfs.DFSClient: DataStreamer Exception java.io.IOException: Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try. (Nodes: current=[172.19.0.3:50010], original=[172.19.0.3:50010]). The current failed datanode replacement policy is DEFAULT, and a client may configure this via 'dfs.client.block.write.replace-datanode-on-failure.policy' in its configuration. at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.findNewDatanode(DFSOutputStream.java:1036) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.addDatanode2ExistingPipeline(DFSOutputStream.java:1102) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1249) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:593) FSDataOutputStream#close error: java.io.IOException: Failed to replace a bad datanode on the existing pipeline due to no more good datanodes being available to try. (Nodes: current=[172.19.0.3:50010], original=[172.19.0.3:50010]). The current failed datanode replacement policy is DEFAULT, and a client may configure this via 'dfs.client.block.write.replace-datanode-on-failure.policy' in its configuration. at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.findNewDatanode(DFSOutputStream.java:1036) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.addDatanode2ExistingPipeline(DFSOutputStream.java:1102) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.setupPipelineForAppendOrRecovery(DFSOutputStream.java:1249) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:593) WARNING: Logging before InitGoogleLogging() is written to STDERR F1217 21:06:38.871151 25929 hdfs.cc:275] Check failed: impl_->Close().ok() *** Check failure stack trace: *** Aborted (core dumped)