Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-6140

WebHDFS cannot create a file with spaces in the name after HA failover changes.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0, 2.4.0
    • Fix Version/s: 2.4.0
    • Component/s: webhdfs
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Attempting to create a file with spaces in the name through WebHDFS now fails with an IllegalArgumentException while attempting to construct a URI. This had been fixed earlier in HDFS-4944. The failure now is a regression introduced by HDFS-6100.

      1. HDFS-6140.1.patch
        2 kB
        Chris Nauroth

        Issue Links

          Activity

          Chris Nauroth created issue -
          Chris Nauroth made changes -
          Field Original Value New Value
          Assignee Chris Nauroth [ cnauroth ]
          Chris Nauroth made changes -
          Link This issue relates to HDFS-4944 [ HDFS-4944 ]
          Chris Nauroth made changes -
          Link This issue is broken by HDFS-6100 [ HDFS-6100 ]
          Hide
          Chris Nauroth added a comment -

          The attached patch fixes the bug by switching to a different URI constructor that takes care of URI escaping for characters like space. I had written a unit test to cover this in HDFS-4944. Unfortunately, the unit test had a bug, which is why we didn't catch this earlier. I've fixed the test too, now passing real spaces in the path instead of pre-escaping them.

          Show
          Chris Nauroth added a comment - The attached patch fixes the bug by switching to a different URI constructor that takes care of URI escaping for characters like space. I had written a unit test to cover this in HDFS-4944 . Unfortunately, the unit test had a bug, which is why we didn't catch this earlier. I've fixed the test too, now passing real spaces in the path instead of pre-escaping them.
          Chris Nauroth made changes -
          Attachment HDFS-6140.1.patch [ 12636114 ]
          Chris Nauroth made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hide
          Chris Nauroth added a comment -

          I also plan to provide a branch-1 patch. The bug is not present on branch-1, but the faulty test is there. I'd like to correct the test in branch-1 too.

          Show
          Chris Nauroth added a comment - I also plan to provide a branch-1 patch. The bug is not present on branch-1, but the faulty test is there. I'd like to correct the test in branch-1 too.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 patch looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
          Tsz Wo Nicholas Sze made changes -
          Hadoop Flags Reviewed [ 10343 ]
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12636114/HDFS-6140.1.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6466//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6466//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12636114/HDFS-6140.1.patch against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/6466//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/6466//console This message is automatically generated.
          Chris Nauroth made changes -
          Link This issue is related to HDFS-6141 [ HDFS-6141 ]
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #5383 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5383/)
          HDFS-6140. WebHDFS cannot create a file with spaces in the name after HA failover changes. Contributed by Chris Nauroth. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1580173)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #5383 (See https://builds.apache.org/job/Hadoop-trunk-Commit/5383/ ) HDFS-6140 . WebHDFS cannot create a file with spaces in the name after HA failover changes. Contributed by Chris Nauroth. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1580173 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java
          Hide
          Chris Nauroth added a comment -

          I committed this to trunk, branch-2 and branch-2.4.

          I had mentioned that I wanted to patch the test in branch-1 too. It turns out that when I did that, it exposed a different bug that we have in branch-1, but not trunk. Since it's a different bug, I filed HDFS-6141 to address fixing branch-1 separately.

          Show
          Chris Nauroth added a comment - I committed this to trunk, branch-2 and branch-2.4. I had mentioned that I wanted to patch the test in branch-1 too. It turns out that when I did that, it exposed a different bug that we have in branch-1, but not trunk. Since it's a different bug, I filed HDFS-6141 to address fixing branch-1 separately.
          Chris Nauroth made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 3.0.0 [ 12320356 ]
          Fix Version/s 2.4.0 [ 12326143 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #517 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/517/)
          HDFS-6140. WebHDFS cannot create a file with spaces in the name after HA failover changes. Contributed by Chris Nauroth. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1580173)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #517 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/517/ ) HDFS-6140 . WebHDFS cannot create a file with spaces in the name after HA failover changes. Contributed by Chris Nauroth. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1580173 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Hdfs-trunk #1709 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1709/)
          HDFS-6140. WebHDFS cannot create a file with spaces in the name after HA failover changes. Contributed by Chris Nauroth. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1580173)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-Hdfs-trunk #1709 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1709/ ) HDFS-6140 . WebHDFS cannot create a file with spaces in the name after HA failover changes. Contributed by Chris Nauroth. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1580173 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java
          Hide
          Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1734 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1734/)
          HDFS-6140. WebHDFS cannot create a file with spaces in the name after HA failover changes. Contributed by Chris Nauroth. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1580173)

          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java
          • /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java
          Show
          Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #1734 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1734/ ) HDFS-6140 . WebHDFS cannot create a file with spaces in the name after HA failover changes. Contributed by Chris Nauroth. (cnauroth: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1580173 ) /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/web/resources/DatanodeWebHdfsMethods.java /hadoop/common/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/web/TestWebHdfsFileSystemContract.java
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Allen Wittenauer made changes -
          Fix Version/s 3.0.0 [ 12320356 ]

            People

            • Assignee:
              Chris Nauroth
              Reporter:
              Chris Nauroth
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development