Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: mrv2
    • Labels:
      None

      Description

      when doing requesting info from the web services rest API, curl seems to return content-type of both text and json or xml:

      > Accept: application/xml
      >
      < HTTP/1.1 200 OK
      < Content-Type: text/plain; charset=utf-8
      < Content-Type: application/xml

        Activity

        Hide
        Thomas Graves added a comment -

        This appears to be that the common HTTPServer class is setting the content-type in the doFilter routine first, then later jersey sets it to what it actually is. Instead of over writing it, it appears to append it. Need to investigate fix.

        Show
        Thomas Graves added a comment - This appears to be that the common HTTPServer class is setting the content-type in the doFilter routine first, then later jersey sets it to what it actually is. Instead of over writing it, it appears to append it. Need to investigate fix.
        Hide
        Thomas Graves added a comment -

        gets rid of the extra content-type header added by the generic HttpServer.

        Show
        Thomas Graves added a comment - gets rid of the extra content-type header added by the generic HttpServer.
        Hide
        Thomas Graves added a comment -

        No way to easily unit test this as we would have to start up the entire httpserver with the yarn web app framework and all, so no tests were added. The existing webservice tests will verify nothing was broken. I manually tested all the webservice calls via curl to make sure only the xml/json content type came out.

        Show
        Thomas Graves added a comment - No way to easily unit test this as we would have to start up the entire httpserver with the yarn web app framework and all, so no tests were added. The existing webservice tests will verify nothing was broken. I manually tested all the webservice calls via curl to make sure only the xml/json content type came out.
        Hide
        Robert Joseph Evans added a comment -

        I don't really like the fix because it requires all of the jersey methods to be modified, in exactly the same way, but that is the exact same fix as HDFS-2441 did so I guess it is OK.

        The root of the issue appears to be that HttpServer.QuotingInputFilter is trying to guess what the content type is based off of the URI of the request, and always setting it no matter what. This seems to be so that we can set the charset of the response to be utf-8. This is coupled with Jersey also outputting the content type instead of overwriting the one that was set previously.

        Also it makes it more difficult for them to be called as regular methods. I really would prefer to see something with the server where we could fix some of this in HttpServer itself so that we don't get the automatic content type for web services. But that is likely to be a bigger change and would be a bit more cross project. Sadly I tried upgrading to Jersey 1.12 and it does not fix the issue.

        So reluctantly I give this a +1.

        Show
        Robert Joseph Evans added a comment - I don't really like the fix because it requires all of the jersey methods to be modified, in exactly the same way, but that is the exact same fix as HDFS-2441 did so I guess it is OK. The root of the issue appears to be that HttpServer.QuotingInputFilter is trying to guess what the content type is based off of the URI of the request, and always setting it no matter what. This seems to be so that we can set the charset of the response to be utf-8. This is coupled with Jersey also outputting the content type instead of overwriting the one that was set previously. Also it makes it more difficult for them to be called as regular methods. I really would prefer to see something with the server where we could fix some of this in HttpServer itself so that we don't get the automatic content type for web services. But that is likely to be a bigger change and would be a bit more cross project. Sadly I tried upgrading to Jersey 1.12 and it does not fix the issue. So reluctantly I give this a +1.
        Hide
        Thomas Graves added a comment -

        I agree with you totally. I wanted to change it in HttpServer but saw that as a bigger effort that we could do later.

        Show
        Thomas Graves added a comment - I agree with you totally. I wanted to change it in HttpServer but saw that as a bigger effort that we could do later.
        Hide
        Thomas Graves added a comment -

        thanks for the review Bobby!
        I've committed this to trunk, branch-2, and branch-0.23

        Show
        Thomas Graves added a comment - thanks for the review Bobby! I've committed this to trunk, branch-2, and branch-0.23
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2208 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2208/)
        MAPREDUCE-3613. web service calls header contains 2 content types (tgraves) (Revision 1330560)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2208 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2208/ ) MAPREDUCE-3613 . web service calls header contains 2 content types (tgraves) (Revision 1330560) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2134 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2134/)
        MAPREDUCE-3613. web service calls header contains 2 content types (tgraves) (Revision 1330560)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2134 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2134/ ) MAPREDUCE-3613 . web service calls header contains 2 content types (tgraves) (Revision 1330560) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2151 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2151/)
        MAPREDUCE-3613. web service calls header contains 2 content types (tgraves) (Revision 1330560)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2151 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2151/ ) MAPREDUCE-3613 . web service calls header contains 2 content types (tgraves) (Revision 1330560) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #239 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/239/)
        merge -r 1330560:1330561 from branch-2. FIXES: MAPREDUCE-3613 (Revision 1330562)

        Result = UNSTABLE
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330562
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #239 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/239/ ) merge -r 1330560:1330561 from branch-2. FIXES: MAPREDUCE-3613 (Revision 1330562) Result = UNSTABLE tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330562 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1026 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1026/)
        MAPREDUCE-3613. web service calls header contains 2 content types (tgraves) (Revision 1330560)

        Result = FAILURE
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1026 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1026/ ) MAPREDUCE-3613 . web service calls header contains 2 content types (tgraves) (Revision 1330560) Result = FAILURE tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1061 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1061/)
        MAPREDUCE-3613. web service calls header contains 2 content types (tgraves) (Revision 1330560)

        Result = FAILURE
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1061 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1061/ ) MAPREDUCE-3613 . web service calls header contains 2 content types (tgraves) (Revision 1330560) Result = FAILURE tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java

          People

          • Assignee:
            Thomas Graves
            Reporter:
            Thomas Graves
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development