Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1317

HDFSProxy needs additional changes to work after changes to streamFile servlet in HDFS-1109

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: contrib/hdfsproxy
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Before HDFS-1109, streamFile had filename passed as a parameter in the request. With HDFS-1109, the filename is part of the request path similar to listPaths and data servlets. The AuthorizationFilter in HdfsProxy needs updating to pick up the path from the request path instead of looking for filename parameter.

      1. HDFS-1317-trunk.patch
        1 kB
        Rohini Palaniswamy
      2. HDFS-1317.patch
        2 kB
        Rohini Palaniswamy

        Issue Links

          Activity

          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 patch looks good.

          Please also provide a patch for Apache 0.22.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 patch looks good. Please also provide a patch for Apache 0.22.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Should the fixed version be 0.22 since HDFS-1109 is only committed to 0.22?

          Show
          Tsz Wo Nicholas Sze added a comment - Should the fixed version be 0.22 since HDFS-1109 is only committed to 0.22?
          Hide
          Rohini Palaniswamy added a comment -

          Patch for apache trunk.

          Show
          Rohini Palaniswamy added a comment - Patch for apache trunk.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 patch looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12450465/HDFS-1317-trunk.patch
          against trunk revision 967293.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12450465/HDFS-1317-trunk.patch against trunk revision 967293. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/console This message is automatically generated.
          Hide
          Rohini Palaniswamy added a comment -

          All the failed tests are not caused by this patch. The related JIRA are listed below.

          TestBlockToken https://issues.apache.org/jira/browse/HDFS-1284
          TestFileAppend4.testRecoverFinalizedBlock https://issues.apache.org/jira/browse/HDFS-1306
          TestFileAppend4.testCompleteOtherLeaseHoldersFile https://issues.apache.org/jira/browse/HDFS-1306
          TestHdfsProxy https://issues.apache.org/jira/browse/HDFS-1301

          https://issues.apache.org/jira/browse/HDFS-1301 was not committed to trunk and it needs to be done. But even with that patch TestHdfsProxy is failing for me with "unauthorized for user x via x". Investigating it.

          Show
          Rohini Palaniswamy added a comment - All the failed tests are not caused by this patch. The related JIRA are listed below. TestBlockToken https://issues.apache.org/jira/browse/HDFS-1284 TestFileAppend4.testRecoverFinalizedBlock https://issues.apache.org/jira/browse/HDFS-1306 TestFileAppend4.testCompleteOtherLeaseHoldersFile https://issues.apache.org/jira/browse/HDFS-1306 TestHdfsProxy https://issues.apache.org/jira/browse/HDFS-1301 https://issues.apache.org/jira/browse/HDFS-1301 was not committed to trunk and it needs to be done. But even with that patch TestHdfsProxy is failing for me with "unauthorized for user x via x". Investigating it.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > https://issues.apache.org/jira/browse/HDFS-1301 was not committed to trunk and it needs to be done. But even with that patch TestHdfsProxy is failing for me with "unauthorized for user x via x". Investigating it.

          Could you follow up on fixing TestHdfsProxy?

          Show
          Tsz Wo Nicholas Sze added a comment - > https://issues.apache.org/jira/browse/HDFS-1301 was not committed to trunk and it needs to be done. But even with that patch TestHdfsProxy is failing for me with "unauthorized for user x via x". Investigating it. Could you follow up on fixing TestHdfsProxy?
          Hide
          Tsz Wo Nicholas Sze added a comment -

          There is also a -1 on the tests.
          "Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch."

          Show
          Tsz Wo Nicholas Sze added a comment - There is also a -1 on the tests. "Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch."
          Hide
          Rohini Palaniswamy added a comment -

          New test was not added for this patch, as changes were made to the Servlet filter. Writing a unit test involved stubbing out HttpServletRequest and other servlet-api classes which is not worth the effort for the minor change of changing a regex pattern to be in sync with the changed contract. Manually verified by deploying hdfsproxy on tomcat and tested that streamFile works.

          Show
          Rohini Palaniswamy added a comment - New test was not added for this patch, as changes were made to the Servlet filter. Writing a unit test involved stubbing out HttpServletRequest and other servlet-api classes which is not worth the effort for the minor change of changing a regex pattern to be in sync with the changed contract. Manually verified by deploying hdfsproxy on tomcat and tested that streamFile works.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this. Thanks, Rohini!

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Rohini!
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #356 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/356/)
          HDFS-1317. Remove the FILEPATH_PATTERN from hdfsproxy.AuthorizationFilter. Contributed by Rohini Palaniswamy

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #356 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/356/ ) HDFS-1317 . Remove the FILEPATH_PATTERN from hdfsproxy.AuthorizationFilter. Contributed by Rohini Palaniswamy

            People

            • Assignee:
              Rohini Palaniswamy
              Reporter:
              Rohini Palaniswamy
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development