Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: mrv2
    • Labels:
      None

      Description

      when doing requesting info from the web services rest API, curl seems to return content-type of both text and json or xml:

      > Accept: application/xml
      >
      < HTTP/1.1 200 OK
      < Content-Type: text/plain; charset=utf-8
      < Content-Type: application/xml

        Activity

        Thomas Graves created issue -
        Thomas Graves made changes -
        Field Original Value New Value
        Parent MAPREDUCE-2863 [ 12519437 ]
        Issue Type Bug [ 1 ] Sub-task [ 7 ]
        Hide
        Thomas Graves added a comment -

        This appears to be that the common HTTPServer class is setting the content-type in the doFilter routine first, then later jersey sets it to what it actually is. Instead of over writing it, it appears to append it. Need to investigate fix.

        Show
        Thomas Graves added a comment - This appears to be that the common HTTPServer class is setting the content-type in the doFilter routine first, then later jersey sets it to what it actually is. Instead of over writing it, it appears to append it. Need to investigate fix.
        Arun C Murthy made changes -
        Priority Blocker [ 1 ] Critical [ 2 ]
        Hide
        Thomas Graves added a comment -

        gets rid of the extra content-type header added by the generic HttpServer.

        Show
        Thomas Graves added a comment - gets rid of the extra content-type header added by the generic HttpServer.
        Thomas Graves made changes -
        Attachment MAPREDUCE-3613.patch [ 12524320 ]
        Thomas Graves made changes -
        Target Version/s 0.23.0 [ 12315570 ] 0.23.3 [ 12320060 ]
        Hide
        Thomas Graves added a comment -

        No way to easily unit test this as we would have to start up the entire httpserver with the yarn web app framework and all, so no tests were added. The existing webservice tests will verify nothing was broken. I manually tested all the webservice calls via curl to make sure only the xml/json content type came out.

        Show
        Thomas Graves added a comment - No way to easily unit test this as we would have to start up the entire httpserver with the yarn web app framework and all, so no tests were added. The existing webservice tests will verify nothing was broken. I manually tested all the webservice calls via curl to make sure only the xml/json content type came out.
        Thomas Graves made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Robert Joseph Evans added a comment -

        I don't really like the fix because it requires all of the jersey methods to be modified, in exactly the same way, but that is the exact same fix as HDFS-2441 did so I guess it is OK.

        The root of the issue appears to be that HttpServer.QuotingInputFilter is trying to guess what the content type is based off of the URI of the request, and always setting it no matter what. This seems to be so that we can set the charset of the response to be utf-8. This is coupled with Jersey also outputting the content type instead of overwriting the one that was set previously.

        Also it makes it more difficult for them to be called as regular methods. I really would prefer to see something with the server where we could fix some of this in HttpServer itself so that we don't get the automatic content type for web services. But that is likely to be a bigger change and would be a bit more cross project. Sadly I tried upgrading to Jersey 1.12 and it does not fix the issue.

        So reluctantly I give this a +1.

        Show
        Robert Joseph Evans added a comment - I don't really like the fix because it requires all of the jersey methods to be modified, in exactly the same way, but that is the exact same fix as HDFS-2441 did so I guess it is OK. The root of the issue appears to be that HttpServer.QuotingInputFilter is trying to guess what the content type is based off of the URI of the request, and always setting it no matter what. This seems to be so that we can set the charset of the response to be utf-8. This is coupled with Jersey also outputting the content type instead of overwriting the one that was set previously. Also it makes it more difficult for them to be called as regular methods. I really would prefer to see something with the server where we could fix some of this in HttpServer itself so that we don't get the automatic content type for web services. But that is likely to be a bigger change and would be a bit more cross project. Sadly I tried upgrading to Jersey 1.12 and it does not fix the issue. So reluctantly I give this a +1.
        Hide
        Thomas Graves added a comment -

        I agree with you totally. I wanted to change it in HttpServer but saw that as a bigger effort that we could do later.

        Show
        Thomas Graves added a comment - I agree with you totally. I wanted to change it in HttpServer but saw that as a bigger effort that we could do later.
        Hide
        Thomas Graves added a comment -

        thanks for the review Bobby!
        I've committed this to trunk, branch-2, and branch-0.23

        Show
        Thomas Graves added a comment - thanks for the review Bobby! I've committed this to trunk, branch-2, and branch-0.23
        Thomas Graves made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.23.3 [ 12320060 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2208 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2208/)
        MAPREDUCE-3613. web service calls header contains 2 content types (tgraves) (Revision 1330560)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2208 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2208/ ) MAPREDUCE-3613 . web service calls header contains 2 content types (tgraves) (Revision 1330560) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2134 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2134/)
        MAPREDUCE-3613. web service calls header contains 2 content types (tgraves) (Revision 1330560)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2134 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2134/ ) MAPREDUCE-3613 . web service calls header contains 2 content types (tgraves) (Revision 1330560) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2151 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2151/)
        MAPREDUCE-3613. web service calls header contains 2 content types (tgraves) (Revision 1330560)

        Result = SUCCESS
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2151 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2151/ ) MAPREDUCE-3613 . web service calls header contains 2 content types (tgraves) (Revision 1330560) Result = SUCCESS tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-0.23-Build #239 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/239/)
        merge -r 1330560:1330561 from branch-2. FIXES: MAPREDUCE-3613 (Revision 1330562)

        Result = UNSTABLE
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330562
        Files :

        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java
        • /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-0.23-Build #239 (See https://builds.apache.org/job/Hadoop-Hdfs-0.23-Build/239/ ) merge -r 1330560:1330561 from branch-2. FIXES: MAPREDUCE-3613 (Revision 1330562) Result = UNSTABLE tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330562 Files : /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java /hadoop/common/branches/branch-0.23/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #1026 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1026/)
        MAPREDUCE-3613. web service calls header contains 2 content types (tgraves) (Revision 1330560)

        Result = FAILURE
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1026 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1026/ ) MAPREDUCE-3613 . web service calls header contains 2 content types (tgraves) (Revision 1330560) Result = FAILURE tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #1061 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1061/)
        MAPREDUCE-3613. web service calls header contains 2 content types (tgraves) (Revision 1330560)

        Result = FAILURE
        tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1061 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1061/ ) MAPREDUCE-3613 . web service calls header contains 2 content types (tgraves) (Revision 1330560) Result = FAILURE tgraves : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1330560 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/webapp/NMWebServices.java /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/webapp/RMWebServices.java
        Arun C Murthy made changes -
        Fix Version/s 2.0.2-alpha [ 12322471 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        113d 40m 1 Thomas Graves 25/Apr/12 19:51
        Patch Available Patch Available Resolved Resolved
        2h 24m 1 Thomas Graves 25/Apr/12 22:15
        Resolved Resolved Closed Closed
        168d 20h 33m 1 Arun C Murthy 11/Oct/12 18:48

          People

          • Assignee:
            Thomas Graves
            Reporter:
            Thomas Graves
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development