Details
-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 1.0.0, 2.0.0-alpha
-
Fix Version/s: 1.1.0, 2.0.2-alpha
-
Component/s: webhdfs
-
Labels:None
-
Hadoop Flags:Reviewed
Description
CREATE currently redirects client to a random datanode but not using the client location information.
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
Nicholas, took a quick look at the patch. It looks good. Can you please add some tests?
h3551_20120625: adds a test
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533437/h3551_20120625.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 javadoc. The javadoc tool appears to have generated 2 warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:
org.apache.hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2701//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2701//console
This message is automatically generated.
The test failure and javadoc warnings are not related to the patch.
Minor comment - javadoc for TestWebHdfsDataLocality says "Test balander...". Also NamenodeProtocols import is not needed.
+1 for the change. It would be good to get this into Release 1.1.
Thanks Suresh for the review.
h3551_20120626.patch: cleans up the imports and rewrites javadoc/comments.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533573/h3551_20120626.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 javadoc. The javadoc tool appears to have generated 2 warning messages.
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2705//testReport/
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2705//console
This message is automatically generated.
Integrated in Hadoop-Mapreduce-trunk-Commit #2411 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2411/)
HDFS-3551. WebHDFS CREATE should use client location for HTTP redirection. (Revision 1354316)
Result = FAILURE
szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1354316
Files :
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/Host2NodesMap.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web/resources
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web/resources/TestWebHdfsDataLocality.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/WebHdfsTestUtil.java
Integrated in Hadoop-Common-trunk-Commit #2392 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2392/)
HDFS-3551. WebHDFS CREATE should use client location for HTTP redirection. (Revision 1354316)
Result = SUCCESS
szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1354316
Files :
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/Host2NodesMap.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web/resources
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web/resources/TestWebHdfsDataLocality.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/WebHdfsTestUtil.java
Integrated in Hadoop-Hdfs-trunk-Commit #2461 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2461/)
HDFS-3551. WebHDFS CREATE should use client location for HTTP redirection. (Revision 1354316)
Result = SUCCESS
szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1354316
Files :
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/Host2NodesMap.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web/resources
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web/resources/TestWebHdfsDataLocality.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/WebHdfsTestUtil.java
Integrated in Hadoop-Hdfs-trunk #1089 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1089/)
HDFS-3551. WebHDFS CREATE should use client location for HTTP redirection. (Revision 1354316)
Result = FAILURE
szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1354316
Files :
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/Host2NodesMap.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web/resources
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web/resources/TestWebHdfsDataLocality.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/WebHdfsTestUtil.java
h3551_20120626_b-1.patch: for branch-1.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12533650/h3551_20120626_b-1.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified test files.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2712//console
This message is automatically generated.
Nicholas, do you know what the following FAILURE means?
Integrated in Hadoop-Mapreduce-trunk-Commit #2411 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2411/) HDFS-3551. WebHDFS CREATE should use client location for HTTP redirection. (Revision 1354316) Result = FAILURE
+1 for the 1.x patch.
We should see if this can make it into 1.1.
Integrated in Hadoop-Mapreduce-trunk #1122 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1122/)
HDFS-3551. WebHDFS CREATE should use client location for HTTP redirection. (Revision 1354316)
Result = FAILURE
szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1354316
Files :
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/Host2NodesMap.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web/resources
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/web/resources/TestWebHdfsDataLocality.java
- /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/WebHdfsTestUtil.java
> Nicholas, do you know what the following FAILURE means?
There was a test failed so that all Hadoop-Mapreduce-trunk-Commit builds failed.
Tests in error: testGetInvalidJob(org.apache.hadoop.mapreduce.v2.TestNonExistentJob) Tests run: 424, Failures: 0, Errors: 1, Skipped: 15 [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] hadoop-yarn ....................................... SUCCESS [2.012s] [INFO] hadoop-yarn-api ................................... SUCCESS [8.148s] [INFO] hadoop-yarn-common ................................ SUCCESS [38.307s] [INFO] hadoop-yarn-server ................................ SUCCESS [0.112s] [INFO] hadoop-yarn-server-common ......................... SUCCESS [4.902s] [INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [2:29.220s] [INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [2.689s] [INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [3:38.993s] [INFO] hadoop-yarn-server-tests .......................... SUCCESS [41.438s] [INFO] hadoop-mapreduce-client ........................... SUCCESS [0.066s] [INFO] hadoop-mapreduce-client-core ...................... SUCCESS [47.188s] [INFO] hadoop-yarn-applications .......................... SUCCESS [0.090s] [INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [23.624s] [INFO] hadoop-yarn-site .................................. SUCCESS [0.240s] [INFO] hadoop-mapreduce-client-common .................... SUCCESS [20.709s] [INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [2.386s] [INFO] hadoop-mapreduce-client-app ....................... SUCCESS [4:33.846s] [INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [1:08.492s] [INFO] hadoop-mapreduce-client-jobclient ................. FAILURE [49:39.662s] [INFO] Apache Hadoop MapReduce Examples .................. SKIPPED [INFO] hadoop-mapreduce .................................. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------
I have committed this.
h3551_20120620.patch: 1st patch; need to add a test.