Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-9244

Upgrade servlet-api dependency from version 2.5 to 3.0.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: 2.0.3-alpha
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Please update the servlet-api jar from 2.5 to javax.servlet 3.0 via Maven:

      <dependency>
      <groupId>javax.servlet</groupId>
      <artifactId>javax.servlet-api</artifactId>
      <version>3.0.1</version>
      <scope>provided</scope>
      </dependency>

      I am running a 2.0.3 dev-cluster and can confirm compatibility. I have removed the servlet-api-2.5.jar file and replaced it with javax.servlet-3.0.jar file. I am using javax.servlet-3.0 because it implements methods that I use for a filter, namely the HttpServletResponse.getStatus() method.

      I believe it is a gain to have this dependency as it allows more functionality and has so far proven to be backwards compatible.

      1. HDFS-4422.patch
        5 kB
        Plamen Jeliazkov

        Issue Links

          Activity

          Hide
          rkanter Robert Kanter added a comment -

          I've been working on HADOOP-10075 to update Jetty 6.x to 9.x, and part of that is updating servlet-api, so I'm closing this JIRA as a duplicate of HADOOP-10075.

          Show
          rkanter Robert Kanter added a comment - I've been working on HADOOP-10075 to update Jetty 6.x to 9.x, and part of that is updating servlet-api, so I'm closing this JIRA as a duplicate of HADOOP-10075 .
          Hide
          aw Allen Wittenauer added a comment - - edited

          Cancelling patch, as it no longer works as expected.

          Show
          aw Allen Wittenauer added a comment - - edited Cancelling patch, as it no longer works as expected.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12565588/HDFS-4422.patch
          against trunk revision da2fb2b.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javac. The patch appears to cause the build to fail.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5623//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12565588/HDFS-4422.patch against trunk revision da2fb2b. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac . The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5623//console This message is automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12565588/HDFS-4422.patch
          against trunk revision 14e2639.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javac. The patch appears to cause the build to fail.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4718//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12565588/HDFS-4422.patch against trunk revision 14e2639. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac . The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/4718//console This message is automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          This is part of a combined update relating to HADOOP-9613 -jersey, HADOOP-10076 - tomcat, & HADOOP-10075 - jetty.

          I don't think we should update things in isolation, especially the servlet interface definition which is not completely implemented in Jetty 6.

          Maybe we should create a single JIRA to cover the update "move the servlet API and implementations up". Then update in trunk & decide whether or not to backport. Now that MapReduce doesn't use Jetty, Hadoop should be less sensitive to these dependencies being changed

          Show
          stevel@apache.org Steve Loughran added a comment - This is part of a combined update relating to HADOOP-9613 -jersey, HADOOP-10076 - tomcat, & HADOOP-10075 - jetty. I don't think we should update things in isolation, especially the servlet interface definition which is not completely implemented in Jetty 6 . Maybe we should create a single JIRA to cover the update "move the servlet API and implementations up". Then update in trunk & decide whether or not to backport. Now that MapReduce doesn't use Jetty, Hadoop should be less sensitive to these dependencies being changed
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Link into the whole "update the POMs" JIRA

          Show
          stevel@apache.org Steve Loughran added a comment - Link into the whole "update the POMs" JIRA
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12565588/HDFS-4422.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javac. The patch appears to cause the build to fail.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3625//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12565588/HDFS-4422.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac . The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/3625//console This message is automatically generated.
          Hide
          wheat9 Haohui Mai added a comment -

          +1 on moving towards servlet 3.0. Servlet 3.0 allows specifies HttpOnly and Secure flags in the cookie, which is required by HADOOP-10379.

          I agree that mixing servlet 3.0 and jetty 6 might confuse the developers, but I think upgrading servlet and jetty can be independently addressed.

          Show
          wheat9 Haohui Mai added a comment - +1 on moving towards servlet 3.0. Servlet 3.0 allows specifies HttpOnly and Secure flags in the cookie, which is required by HADOOP-10379 . I agree that mixing servlet 3.0 and jetty 6 might confuse the developers, but I think upgrading servlet and jetty can be independently addressed.
          Hide
          sureshms Suresh Srinivas added a comment -

          Looks like the consensus is not to upgrade the jetty in branch-2. So, shall we move forward with Jetty8 and servlet3 in the trunk?

          +1

          Does any one know performance comparison of Jetty 8 with that of Jetty 6? During WebHDFS work, performance analysis indicated poor performance/latency in http layers (did not spend too much confirming the same). I was interested in seeing if later versions of Jetty would result in better performance. Given that Jetty is becoming critical not only for REST APIs, but also for data paths, understanding the performance gains/losses would be great.

          Show
          sureshms Suresh Srinivas added a comment - Looks like the consensus is not to upgrade the jetty in branch-2. So, shall we move forward with Jetty8 and servlet3 in the trunk? +1 Does any one know performance comparison of Jetty 8 with that of Jetty 6? During WebHDFS work, performance analysis indicated poor performance/latency in http layers (did not spend too much confirming the same). I was interested in seeing if later versions of Jetty would result in better performance. Given that Jetty is becoming critical not only for REST APIs, but also for data paths, understanding the performance gains/losses would be great.
          Hide
          cos Konstantin Boudnik added a comment -

          Looks like the consensus is not to upgrade the jetty in branch-2. So, shall we move forward with Jetty8 and servlet3 in the trunk?

          Show
          cos Konstantin Boudnik added a comment - Looks like the consensus is not to upgrade the jetty in branch-2. So, shall we move forward with Jetty8 and servlet3 in the trunk?
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12565588/HDFS-4422.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javac. The applied patch generated 2024 javac compiler warnings (more than the trunk's current 2022 warnings).

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-client hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-auth-examples hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2087//testReport/
          Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/2087//artifact/trunk/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2087//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12565588/HDFS-4422.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac . The applied patch generated 2024 javac compiler warnings (more than the trunk's current 2022 warnings). +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-client hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-auth-examples hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/2087//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/2087//artifact/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/2087//console This message is automatically generated.
          Hide
          zero45 Plamen Jeliazkov added a comment -

          Steve, Boudnik: Is it not Jetty 8 that is supposed to be fully compatible with Servlet API 3.0? Not Jetty 7? Perhaps that will be too far foward.

          Show
          zero45 Plamen Jeliazkov added a comment - Steve, Boudnik: Is it not Jetty 8 that is supposed to be fully compatible with Servlet API 3.0? Not Jetty 7? Perhaps that will be too far foward.
          Hide
          zero45 Plamen Jeliazkov added a comment -

          Suresh: I have ran the full unit test suite locally as well as operate a distributed cluster which uses the 3.0 API in custom filters. I have not seen any disruptions and / or drops of any of the HTTP servers with the dependency upgrade installed and usage of the cluster. I have not tested with MapReduce / YARN however.

          All: If you are more comfortable with an upgrade to jetty7 and servlet3 that is fine.

          Show
          zero45 Plamen Jeliazkov added a comment - Suresh: I have ran the full unit test suite locally as well as operate a distributed cluster which uses the 3.0 API in custom filters. I have not seen any disruptions and / or drops of any of the HTTP servers with the dependency upgrade installed and usage of the cluster. I have not tested with MapReduce / YARN however. All: If you are more comfortable with an upgrade to jetty7 and servlet3 that is fine.
          Hide
          cos Konstantin Boudnik added a comment -

          I've moved the issues to Common project.

          As for the combined switch of the Servlet API 3.0 + Jetty7: I am for it, but considering the stable state of 2.0.3 it might be somewhat a disruptive change; but it seems totally fine for 3.0 line considering the time frames, etc. Don't you think?

          Show
          cos Konstantin Boudnik added a comment - I've moved the issues to Common project. As for the combined switch of the Servlet API 3.0 + Jetty7: I am for it, but considering the stable state of 2.0.3 it might be somewhat a disruptive change; but it seems totally fine for 3.0 line considering the time frames, etc. Don't you think?
          Hide
          stevel@apache.org Steve Loughran added a comment -

          thinking about this some more, I'm not convinced that you should mix jetty6 & servlet3. People are free to do that themselves, but as it doesn't look like 100% supported in jetty6, a move to servlet3 in the hadoop build which I'm not against should only happen with a switch to jetty7.

          Show
          stevel@apache.org Steve Loughran added a comment - thinking about this some more, I'm not convinced that you should mix jetty6 & servlet3. People are free to do that themselves, but as it doesn't look like 100% supported in jetty6, a move to servlet3 in the hadoop build which I'm not against should only happen with a switch to jetty7.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Before we move, I've just been looking at jetty + servlet API 3 details on the web

          2008 ref to jetty 7: http://docs.codehaus.org/display/JETTY/Continuations

          jetty 7 + 8 details: http://wiki.eclipse.org/Jetty/Feature/Continuations

          the implication here is that continuations (which allow you to release/resume a request, have the servlet engine push the request to some stack and pop it later) in the jetty6 API will work on jetty 7+ 8, but there's an implict hint here that the servlet 3.0 equivalent only work on jetty7+.

          That is: not everything in the 3.0 servlet API is in Jetty6.

          If there are things in the 3.0 API that people need to use, they should have explicit unit tests to verify they work, and everyone should know which features of the API are missing -& where you should fallback to jetty-specific details

          Show
          stevel@apache.org Steve Loughran added a comment - Before we move, I've just been looking at jetty + servlet API 3 details on the web 2008 ref to jetty 7: http://docs.codehaus.org/display/JETTY/Continuations jetty 7 + 8 details: http://wiki.eclipse.org/Jetty/Feature/Continuations the implication here is that continuations (which allow you to release/resume a request, have the servlet engine push the request to some stack and pop it later) in the jetty6 API will work on jetty 7+ 8, but there's an implict hint here that the servlet 3.0 equivalent only work on jetty7+. That is: not everything in the 3.0 servlet API is in Jetty6. If there are things in the 3.0 API that people need to use, they should have explicit unit tests to verify they work, and everyone should know which features of the API are missing -& where you should fallback to jetty-specific details
          Hide
          sureshms Suresh Srinivas added a comment -

          I am running a 2.0.3 dev-cluster and can confirm compatibility.

          What kind of tests did you run to confirm compatibility?

          I am thinking if this JIRA has to be in hadoop-common rather than here?

          Agree. This should be moved hadoop-common.

          Show
          sureshms Suresh Srinivas added a comment - I am running a 2.0.3 dev-cluster and can confirm compatibility. What kind of tests did you run to confirm compatibility? I am thinking if this JIRA has to be in hadoop-common rather than here? Agree. This should be moved hadoop-common.
          Hide
          cos Konstantin Boudnik added a comment -

          I am thinking if this JIRA has to be in hadoop-common rather than here?

          Show
          cos Konstantin Boudnik added a comment - I am thinking if this JIRA has to be in hadoop-common rather than here?
          Hide
          cos Konstantin Boudnik added a comment -

          It is actually seems to be an independent action. Looks like Jetty 6 works with 3.0 version of the servlet API.

          Show
          cos Konstantin Boudnik added a comment - It is actually seems to be an independent action. Looks like Jetty 6 works with 3.0 version of the servlet API.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Can you update the servlet API without updating Jetty, or is it a combined action?

          Show
          stevel@apache.org Steve Loughran added a comment - Can you update the servlet API without updating Jetty, or is it a combined action?
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12565588/HDFS-4422.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          -1 javac. The applied patch generated 2023 javac compiler warnings (more than the trunk's current 2021 warnings).

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-client hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-auth-examples hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3860//testReport/
          Javac warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/3860//artifact/trunk/patchprocess/diffJavacWarnings.txt
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3860//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12565588/HDFS-4422.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac . The applied patch generated 2023 javac compiler warnings (more than the trunk's current 2021 warnings). +1 javadoc . The javadoc tool did not generate any warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-client hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-auth-examples hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-httpfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy. +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/3860//testReport/ Javac warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/3860//artifact/trunk/patchprocess/diffJavacWarnings.txt Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/3860//console This message is automatically generated.
          Hide
          cos Konstantin Boudnik added a comment -

          Considering the benign nature of the patch and backward compatibility between the API versions it seems like a sensible thing to be included into the 2.0.3 release? What do you think Arun?

          Show
          cos Konstantin Boudnik added a comment - Considering the benign nature of the patch and backward compatibility between the API versions it seems like a sensible thing to be included into the 2.0.3 release? What do you think Arun?
          Hide
          zero45 Plamen Jeliazkov added a comment -

          Patch for review.

          Show
          zero45 Plamen Jeliazkov added a comment - Patch for review.

            People

            • Assignee:
              zero45 Plamen Jeliazkov
              Reporter:
              zero45 Plamen Jeliazkov
            • Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development