Harmony
  1. Harmony
  2. HARMONY-6452

HttpUrlConnection converts request headers to lowercase - HttpUrlConnection.addRequestProperty overrides existing properties

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.0M1, 5.0M13
    • Component/s: Classlib
    • Labels:
      None
    • Environment:
      Android 1.6
    • Patch Info:
      Patch Available

      Description

      Problem: Lower-Case-Headers
      ==============================
      In Sun's JRE this code
      java.net.HttpURLConnection conn = (java.net.HttpURLConnection)(new java.net.URL("http://...").openConnection());
      conn.setRequestProperty("Content-Type", "multipart/form-data; boundary=...");
      System.out.println (conn.getRequestProperty("content-type"));
      System.out.println (conn.getRequestProperties());
      produces the following output:
      multipart/form-data; boundary=...

      {Content-Type=[multipart/form-data; boundary=...]}

      The Property-Name "Content-Type" is stored case sensitive but it can be fetched in lower-case.
      The resulting Request-Header sent to the Web-Server is case sensitive.

      On Android 1.6 (which uses Apache Harmony) the same Code produces the following output:
      multipart/form-data; boundary=...

      {content-type=[multipart/form-data; boundary=...]}

      This time the Property-Name "Content-Type" is stored in lower-case. The resulting Request-Header is also sent in lower-case. That's a violation of the HTTP 1.1 spec, and certain service providers may ignore such Request-Headers.

      In harmony 5.0 r901653 the output is the same as on Android 1.6 but the resulting Request-Header is sent case-sensitive.

      Problem addRequestProperty overrides existing properties
      ==============================================
      Existing Properties should be appended.

      Example
      ==============================================
      HttpURLConnection conn = (HttpURLConnection)(new URL("http://www.example.com/index.html").openConnection());
      conn.setRequestProperty("Content-Disposition", "Content-Disposition: form-data");
      conn.addRequestProperty("Content-Disposition", "name=\"file\"");
      conn.addRequestProperty("Content-Disposition", "filename=\"test.html\"");
      conn.setRequestProperty("a", "1");
      conn.addRequestProperty("a", "2");
      System.out.println ("Single Property: "+conn.getRequestProperty("content-disposition"));
      System.out.println ("All Properties:");
      for (Map.Entry<String, List<String>> entry : conn.getRequestProperties().entrySet()) {
      System.out.print (" Property: "entry.getKey()" =

      {"); for (String v : entry.getValue()) System.out.print (v+";"); System.out.println ("}

      ");
      }
      Should produce the following output:
      Single Property: filename="test.html"
      All Properties:
      Property: a =

      {2;1;}

      Property: Content-Disposition =

      {filename="test.html";name="file";Content-Disposition: form-data;}

      But the following output is produced:
      Single Property: filename="test.html"
      All Properties:
      Property: content-disposition =

      {filename="test.html";}

      Property: a =

      {2;}

      I have set the Property "a" in lower-case to make clear, that this problem is depends on the lower-case headers problem.

      Info
      ==============================================
      The class org.apache.harmony.luni.internal.net.www.protocol.http.Header is not correct implemented. It only works with lower-case property-names and each property-value can only contain one entry.

      Question
      ==============================
      1. Does anybody know how to find out which version of harmony is used by Android 1.6? I tried System.getProperties() but that didn't help.

      1. descriptions.txt
        2 kB
        Michael Andresen
      2. Header.java
        8 kB
        Michael Andresen

        Activity

        Hide
        Mark Hindess added a comment -

        On the left of the web page for this JIRA, there should be an "Attach file" item under the "Operations" section on the left-hand side. You should use this to attach a patch ... be sure to tick the "Grant license to ASF for inclusion in ASF works" in order that we may use your contribution and it might be an idea to read:

        http://harmony.apache.org/contribution_policy.html

        Thanks,
        Mark

        Show
        Mark Hindess added a comment - On the left of the web page for this JIRA, there should be an "Attach file" item under the "Operations" section on the left-hand side. You should use this to attach a patch ... be sure to tick the "Grant license to ASF for inclusion in ASF works" in order that we may use your contribution and it might be an idea to read: http://harmony.apache.org/contribution_policy.html Thanks, Mark
        Hide
        Elliott Hughes added a comment -

        see also http://code.google.com/p/android/issues/detail?id=6684. i've fixed the case problem in Android, but still need to fill out the Apache paperwork.

        the submitter didn't give an example of "each property-value can only contain one entry" which, unlike the case problem, isn't obviously true.

        Show
        Elliott Hughes added a comment - see also http://code.google.com/p/android/issues/detail?id=6684 . i've fixed the case problem in Android, but still need to fill out the Apache paperwork. the submitter didn't give an example of "each property-value can only contain one entry" which, unlike the case problem, isn't obviously true.
        Hide
        Michael Andresen added a comment -
        • Added an example for the replaced property problem
        • Removed the question howto attach a patch file - thanks to Mark Hindess
        Show
        Michael Andresen added a comment - Added an example for the replaced property problem Removed the question howto attach a patch file - thanks to Mark Hindess
        Hide
        Michael Andresen added a comment -

        Sorry, the replaced-property-problem only occurs when the property-name is mixed case. I've update the example.

        Show
        Michael Andresen added a comment - Sorry, the replaced-property-problem only occurs when the property-name is mixed case. I've update the example.
        Hide
        Michael Andresen added a comment -

        I attached two files:
        Header.java:
        ==========
        This file contains the class org.apache.harmony.luni.internal.net.www.protocol.http.Header.
        I fixed all bugs in it so that the problems regarding this are solved.

        Descriptions:
        ===========
        Documentation about what I've changed compared to the original Header-Class.

        Show
        Michael Andresen added a comment - I attached two files: Header.java: ========== This file contains the class org.apache.harmony.luni.internal.net.www.protocol.http.Header. I fixed all bugs in it so that the problems regarding this are solved. Descriptions: =========== Documentation about what I've changed compared to the original Header-Class.
        Hide
        Mark Hindess added a comment -

        Michael, thanks for your suggested fix. It is preferable to submit a patch rather than a complete replacement - see http://harmony.apache.org/get-involved.html#Guidelines%20on%20how%20to%20Create%20and%20Submit%20a%20Patch - but I'm happy to work with the attached copy for now.

        With your patch some simple cases seem to produce odd behaviour, such as:

        conn.setRequestProperty("KEY", "upper");
        conn.setRequestProperty("key", "lower");
        System.out.println("KEY=" + conn.getRequestProperty("KEY"));
        System.out.println("key=" + conn.getRequestProperty("key"));
        System.out.println("properties=" + conn.getRequestProperties());

        produces output:

        KEY=lower
        key=lower
        properties=

        {KEY=[upper]}

        and also:

        conn.setRequestProperty("key", "lower");
        conn.setRequestProperty("KEY", "upper");
        System.out.println("KEY=" + conn.getRequestProperty("KEY"));
        System.out.println("key=" + conn.getRequestProperty("key"));
        System.out.println("properties=" + conn.getRequestProperties());

        produces:

        KEY=upper
        key=upper
        properties=

        {KEY=[upper], key=[lower]}

        So it is still not quite right. Having said that, I wonder if perhaps we are aiming for the wrong solution here. The reference implementation seems rather inconsistent so I wonder if it is really worth trying to match its behaviour other than meeting the spec.

        I've raised this on the dev list to try to get consensus on the way to fix this and to see if anyone can makes sense of the reference implementation behaviour. See:

        http://markmail.org/thread/3lyqkyg3oxyeithc

        Please join that discussion and we can summarize any conclusions on this JIRA. In the meantime, I'll continue to investigate.

        Show
        Mark Hindess added a comment - Michael, thanks for your suggested fix. It is preferable to submit a patch rather than a complete replacement - see http://harmony.apache.org/get-involved.html#Guidelines%20on%20how%20to%20Create%20and%20Submit%20a%20Patch - but I'm happy to work with the attached copy for now. With your patch some simple cases seem to produce odd behaviour, such as: conn.setRequestProperty("KEY", "upper"); conn.setRequestProperty("key", "lower"); System.out.println("KEY=" + conn.getRequestProperty("KEY")); System.out.println("key=" + conn.getRequestProperty("key")); System.out.println("properties=" + conn.getRequestProperties()); produces output: KEY=lower key=lower properties= {KEY=[upper]} and also: conn.setRequestProperty("key", "lower"); conn.setRequestProperty("KEY", "upper"); System.out.println("KEY=" + conn.getRequestProperty("KEY")); System.out.println("key=" + conn.getRequestProperty("key")); System.out.println("properties=" + conn.getRequestProperties()); produces: KEY=upper key=upper properties= {KEY=[upper], key=[lower]} So it is still not quite right. Having said that, I wonder if perhaps we are aiming for the wrong solution here. The reference implementation seems rather inconsistent so I wonder if it is really worth trying to match its behaviour other than meeting the spec. I've raised this on the dev list to try to get consensus on the way to fix this and to see if anyone can makes sense of the reference implementation behaviour. See: http://markmail.org/thread/3lyqkyg3oxyeithc Please join that discussion and we can summarize any conclusions on this JIRA. In the meantime, I'll continue to investigate.
        Hide
        Mark Hindess added a comment -

        I've committed a fix (and regression test) for item 1 from your descriptions.txt file in r912259. As elliott suggested on the dev@harmony.apache.org mailing list in the above thread, I think using a TreeMap(String.CASE_INSENSITIVE_ORDER) will be most consistent solution for the other problems (and it fixes the inconsistency I mention above with your patch). I'll commit that change too soon.

        Show
        Mark Hindess added a comment - I've committed a fix (and regression test) for item 1 from your descriptions.txt file in r912259. As elliott suggested on the dev@harmony.apache.org mailing list in the above thread, I think using a TreeMap(String.CASE_INSENSITIVE_ORDER) will be most consistent solution for the other problems (and it fixes the inconsistency I mention above with your patch). I'll commit that change too soon.
        Hide
        Hudson added a comment -

        Integrated in Harmony-1.5-head-linux-x86_64 #644 (See http://hudson.zones.apache.org/hudson/job/Harmony-1.5-head-linux-x86_64/644/)
        Fix for item 1 from descriptions.txt of "[#] HttpUrlConnection
        converts request headers to lowercase ...". The props list corruption.

        Show
        Hudson added a comment - Integrated in Harmony-1.5-head-linux-x86_64 #644 (See http://hudson.zones.apache.org/hudson/job/Harmony-1.5-head-linux-x86_64/644/ ) Fix for item 1 from descriptions.txt of " [#] HttpUrlConnection converts request headers to lowercase ...". The props list corruption.
        Hide
        Mark Hindess added a comment -

        I've committed fixes and regression tests for all the issues covered by this JIRA in r912351. Thanks to elliot for his input on the dev@ list.

        Michael, I didn't use your fix but I think I've covered everything with the two commits. Can you confirm that this is the case by closing this JIRA. (Or add further comments if I've missed something.)

        Many thanks,
        Mark.

        Show
        Mark Hindess added a comment - I've committed fixes and regression tests for all the issues covered by this JIRA in r912351. Thanks to elliot for his input on the dev@ list. Michael, I didn't use your fix but I think I've covered everything with the two commits. Can you confirm that this is the case by closing this JIRA. (Or add further comments if I've missed something.) Many thanks, Mark.
        Hide
        Hudson added a comment -

        Integrated in Harmony-1.5-head-linux-x86_64 #645 (See http://hudson.zones.apache.org/hudson/job/Harmony-1.5-head-linux-x86_64/645/)
        Add regression tests and fix remaining issues from "[#]
        HttpUrlConnection converts request headers to lowercase ...".

        Show
        Hudson added a comment - Integrated in Harmony-1.5-head-linux-x86_64 #645 (See http://hudson.zones.apache.org/hudson/job/Harmony-1.5-head-linux-x86_64/645/ ) Add regression tests and fix remaining issues from " [#] HttpUrlConnection converts request headers to lowercase ...".
        Hide
        Michael Andresen added a comment -

        Works great!

        Thanks

        Show
        Michael Andresen added a comment - Works great! Thanks

          People

          • Assignee:
            Mark Hindess
            Reporter:
            Michael Andresen
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development