Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-1317

HDFSProxy needs additional changes to work after changes to streamFile servlet in HDFS-1109

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: contrib/hdfsproxy
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Before HDFS-1109, streamFile had filename passed as a parameter in the request. With HDFS-1109, the filename is part of the request path similar to listPaths and data servlets. The AuthorizationFilter in HdfsProxy needs updating to pick up the path from the request path instead of looking for filename parameter.

      1. HDFS-1317.patch
        2 kB
        Rohini Palaniswamy
      2. HDFS-1317-trunk.patch
        1 kB
        Rohini Palaniswamy

        Issue Links

          Activity

          Rohini Palaniswamy created issue -
          Rohini Palaniswamy made changes -
          Field Original Value New Value
          Attachment HDFS-1317.patch [ 12450275 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 patch looks good.

          Please also provide a patch for Apache 0.22.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 patch looks good. Please also provide a patch for Apache 0.22.
          Tsz Wo Nicholas Sze made changes -
          Link This issue relates to HDFS-1109 [ HDFS-1109 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Should the fixed version be 0.22 since HDFS-1109 is only committed to 0.22?

          Show
          Tsz Wo Nicholas Sze added a comment - Should the fixed version be 0.22 since HDFS-1109 is only committed to 0.22?
          Tsz Wo Nicholas Sze made changes -
          Assignee Rohini Palaniswamy [ rohini ]
          Rohini Palaniswamy made changes -
          Fix Version/s 0.22.0 [ 12314241 ]
          Fix Version/s 0.21.0 [ 12314046 ]
          Hide
          Rohini Palaniswamy added a comment -

          Patch for apache trunk.

          Show
          Rohini Palaniswamy added a comment - Patch for apache trunk.
          Rohini Palaniswamy made changes -
          Attachment HDFS-1317-trunk.patch [ 12450465 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          +1 patch looks good.

          Show
          Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
          Tsz Wo Nicholas Sze made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Hadoop Flags [Reviewed]
          Affects Version/s 0.22.0 [ 12314241 ]
          Affects Version/s 0.21.0 [ 12314046 ]
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12450465/HDFS-1317-trunk.patch
          against trunk revision 967293.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12450465/HDFS-1317-trunk.patch against trunk revision 967293. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/222/console This message is automatically generated.
          Hide
          Rohini Palaniswamy added a comment -

          All the failed tests are not caused by this patch. The related JIRA are listed below.

          TestBlockToken https://issues.apache.org/jira/browse/HDFS-1284
          TestFileAppend4.testRecoverFinalizedBlock https://issues.apache.org/jira/browse/HDFS-1306
          TestFileAppend4.testCompleteOtherLeaseHoldersFile https://issues.apache.org/jira/browse/HDFS-1306
          TestHdfsProxy https://issues.apache.org/jira/browse/HDFS-1301

          https://issues.apache.org/jira/browse/HDFS-1301 was not committed to trunk and it needs to be done. But even with that patch TestHdfsProxy is failing for me with "unauthorized for user x via x". Investigating it.

          Show
          Rohini Palaniswamy added a comment - All the failed tests are not caused by this patch. The related JIRA are listed below. TestBlockToken https://issues.apache.org/jira/browse/HDFS-1284 TestFileAppend4.testRecoverFinalizedBlock https://issues.apache.org/jira/browse/HDFS-1306 TestFileAppend4.testCompleteOtherLeaseHoldersFile https://issues.apache.org/jira/browse/HDFS-1306 TestHdfsProxy https://issues.apache.org/jira/browse/HDFS-1301 https://issues.apache.org/jira/browse/HDFS-1301 was not committed to trunk and it needs to be done. But even with that patch TestHdfsProxy is failing for me with "unauthorized for user x via x". Investigating it.
          Tsz Wo Nicholas Sze made changes -
          Link This issue depends on HDFS-1301 [ HDFS-1301 ]
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > https://issues.apache.org/jira/browse/HDFS-1301 was not committed to trunk and it needs to be done. But even with that patch TestHdfsProxy is failing for me with "unauthorized for user x via x". Investigating it.

          Could you follow up on fixing TestHdfsProxy?

          Show
          Tsz Wo Nicholas Sze added a comment - > https://issues.apache.org/jira/browse/HDFS-1301 was not committed to trunk and it needs to be done. But even with that patch TestHdfsProxy is failing for me with "unauthorized for user x via x". Investigating it. Could you follow up on fixing TestHdfsProxy?
          Hide
          Tsz Wo Nicholas Sze added a comment -

          There is also a -1 on the tests.
          "Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch."

          Show
          Tsz Wo Nicholas Sze added a comment - There is also a -1 on the tests. "Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch."
          Hide
          Rohini Palaniswamy added a comment -

          New test was not added for this patch, as changes were made to the Servlet filter. Writing a unit test involved stubbing out HttpServletRequest and other servlet-api classes which is not worth the effort for the minor change of changing a regex pattern to be in sync with the changed contract. Manually verified by deploying hdfsproxy on tomcat and tested that streamFile works.

          Show
          Rohini Palaniswamy added a comment - New test was not added for this patch, as changes were made to the Servlet filter. Writing a unit test involved stubbing out HttpServletRequest and other servlet-api classes which is not worth the effort for the minor change of changing a regex pattern to be in sync with the changed contract. Manually verified by deploying hdfsproxy on tomcat and tested that streamFile works.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I have committed this. Thanks, Rohini!

          Show
          Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Rohini!
          Tsz Wo Nicholas Sze made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #356 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/356/)
          HDFS-1317. Remove the FILEPATH_PATTERN from hdfsproxy.AuthorizationFilter. Contributed by Rohini Palaniswamy

          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #356 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/356/ ) HDFS-1317 . Remove the FILEPATH_PATTERN from hdfsproxy.AuthorizationFilter. Contributed by Rohini Palaniswamy
          Konstantin Shvachko made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Gavin made changes -
          Link This issue depends on HDFS-1301 [ HDFS-1301 ]
          Gavin made changes -
          Link This issue depends upon HDFS-1301 [ HDFS-1301 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          3d 8h 9m 1 Tsz Wo Nicholas Sze 26/Jul/10 15:42
          Patch Available Patch Available Resolved Resolved
          3d 2h 30m 1 Tsz Wo Nicholas Sze 29/Jul/10 18:13
          Resolved Resolved Closed Closed
          500d 12h 5m 1 Konstantin Shvachko 12/Dec/11 06:18

            People

            • Assignee:
              Rohini Palaniswamy
              Reporter:
              Rohini Palaniswamy
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development