Commons FileUpload
  1. Commons FileUpload
  2. FILEUPLOAD-135

InputStream created with Streaming API returns EOF on first read() for short files uploaded from FireFox over HTTPS

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.2, 1.2.1
    • Fix Version/s: 1.2.1
    • Labels:
      None
    • Environment:

      Windows XP
      Browser: Firefox 1.5.0.11
      Protocol: HTTPS

      Description

      This problem happens only with files shorer then boundary string generated by browser and only with Firefox using HTTPS protocol.
      For some reason in this particular environment inputStream.read() in MultipartStream.ItemInputStream.makeAvailable() reads not whole HTTP response body, but only file content before boundary string.
      I've created a patch fixing this issue.

      1. commons-fileupload-1.1-bug-short-file-eof.patch
        2 kB
        Alexander Sova
      2. commons-fileupload-1.2-bug-short-file-eof.patch
        3 kB
        Alexander Sova
      3. FILEUPLOAD135.patch
        6 kB
        Alexander Sova

        Issue Links

          Activity

          Hide
          Alexander Sova added a comment -

          reading from inputStream until we get enough content to fit boundary string

          Show
          Alexander Sova added a comment - reading from inputStream until we get enough content to fit boundary string
          Hide
          Alexander Sova added a comment -

          my previous patch has a bug. please use new one

          Show
          Alexander Sova added a comment - my previous patch has a bug. please use new one
          Hide
          Jochen Wiedmann added a comment -

          Before applying your patch, I'd like to understand the problem. Would it be possible for you to provide a test case, which fails with 1.2 and works with your patch? Preferrably a patch against the test suite, but a sample http body should do.

          Show
          Jochen Wiedmann added a comment - Before applying your patch, I'd like to understand the problem. Would it be possible for you to provide a test case, which fails with 1.2 and works with your patch? Preferrably a patch against the test suite, but a sample http body should do.
          Hide
          Alexander Sova added a comment -

          The problem is not with content of HTTP response, but with original InputStream behavior. InputStream.read() may read not whole buffer or whole content of input stream, but just part of it. If this part is shorter then boundary string (but still greater then 0) ItemInputStream.read() returns -1.

          I was able to reproduce it only with small file (10 bytes) using FireFox 1.5.0.11 and HTTPS. I think I can simulate this by developing custom InputStream, but it could take some time.

          Here is fragment of my code where I test the upload:

          protected byte[] getBoundary(String contentType)
          {
          ParameterParser parser = new ParameterParser();
          parser.setLowerCaseNames(true);
          // Parameter parser can handle null input
          Map params = parser.parse(contentType, ';');
          String boundaryStr = (String) params.get("boundary");

          if (boundaryStr == null)

          { return null; }

          byte[] boundary;
          try

          { boundary = boundaryStr.getBytes("ISO-8859-1"); }

          catch (UnsupportedEncodingException e)

          { boundary = boundaryStr.getBytes(); }

          return boundary;
          }

          private void testFileUpload(HttpServletRequest req)
          {
          try
          {
          byte[] boundary = getBoundary(req.getHeader("Content-Type"));
          System.out.println("Boundary: " + new String(boundary));
          MultipartStream multipartStream = new MultipartStream(req.getInputStream(), boundary);
          boolean nextPart = multipartStream.skipPreamble();
          OutputStream output = System.out;
          while(nextPart)

          { //header = chunks.readHeader(); System.out.println("!" + multipartStream.readHeaders() + "!"); multipartStream.readBodyData(output); nextPart = multipartStream.readBoundary(); }

          }
          catch(Exception e)

          { e.printStackTrace(); }

          }

          Show
          Alexander Sova added a comment - The problem is not with content of HTTP response, but with original InputStream behavior. InputStream.read() may read not whole buffer or whole content of input stream, but just part of it. If this part is shorter then boundary string (but still greater then 0) ItemInputStream.read() returns -1. I was able to reproduce it only with small file (10 bytes) using FireFox 1.5.0.11 and HTTPS. I think I can simulate this by developing custom InputStream, but it could take some time. Here is fragment of my code where I test the upload: protected byte[] getBoundary(String contentType) { ParameterParser parser = new ParameterParser(); parser.setLowerCaseNames(true); // Parameter parser can handle null input Map params = parser.parse(contentType, ';'); String boundaryStr = (String) params.get("boundary"); if (boundaryStr == null) { return null; } byte[] boundary; try { boundary = boundaryStr.getBytes("ISO-8859-1"); } catch (UnsupportedEncodingException e) { boundary = boundaryStr.getBytes(); } return boundary; } private void testFileUpload(HttpServletRequest req) { try { byte[] boundary = getBoundary(req.getHeader("Content-Type")); System.out.println("Boundary: " + new String(boundary)); MultipartStream multipartStream = new MultipartStream(req.getInputStream(), boundary); boolean nextPart = multipartStream.skipPreamble(); OutputStream output = System.out; while(nextPart) { //header = chunks.readHeader(); System.out.println("!" + multipartStream.readHeaders() + "!"); multipartStream.readBodyData(output); nextPart = multipartStream.readBoundary(); } } catch(Exception e) { e.printStackTrace(); } }
          Hide
          Jochen Wiedmann added a comment -

          This test case doesn't help me understand the problem. The important part is the multipart document, which triggers the problem. We'd need a code sample that creates such a multipart document (compare, for example, the methid StreamingTest.newRequest; btw, note that this method creates file items which are clearly smaller than the boundary) or at least the multipart document itself, which you can catch with tools like tcpmon or Wireshark.

          Show
          Jochen Wiedmann added a comment - This test case doesn't help me understand the problem. The important part is the multipart document, which triggers the problem. We'd need a code sample that creates such a multipart document (compare, for example, the methid StreamingTest.newRequest; btw, note that this method creates file items which are clearly smaller than the boundary) or at least the multipart document itself, which you can catch with tools like tcpmon or Wireshark.
          Hide
          Alexander Sova added a comment -

          I've added new test case to StreamingTest class. The key thing here is defining custom InputStream on top of ByteArrayInputStream which never read more then 3 bytes per call. If you change this number to something bigger then 10, the test will pass.

          BTW, with MockHttpServletRequest.MyServletInputStream.read(byte[],int,int) method defined properly testIOException() start failing. Without this change the second read() method defined in testIOException() never been called.

          Show
          Alexander Sova added a comment - I've added new test case to StreamingTest class. The key thing here is defining custom InputStream on top of ByteArrayInputStream which never read more then 3 bytes per call. If you change this number to something bigger then 10, the test will pass. BTW, with MockHttpServletRequest.MyServletInputStream.read(byte[],int,int) method defined properly testIOException() start failing. Without this change the second read() method defined in testIOException() never been called.
          Hide
          Rob Slifka added a comment -

          Hello Jira,

          I've run into this issue as well, very much looking forward to the
          patch, thanks

          Rob Slifka

          Show
          Rob Slifka added a comment - Hello Jira, I've run into this issue as well, very much looking forward to the patch, thanks Rob Slifka
          Hide
          Rob Slifka added a comment -

          "Hello Jira" yikes

          Anyway, is there a way to see the progress on this issue? I.e. if it's been submitted, scheduled, etc.?

          Just to verify, it is indeed a Firefox-only issue, and not reproducible on any other browser (IE6, IE7, Opera, Safari).

          Show
          Rob Slifka added a comment - "Hello Jira" yikes Anyway, is there a way to see the progress on this issue? I.e. if it's been submitted, scheduled, etc.? Just to verify, it is indeed a Firefox-only issue, and not reproducible on any other browser (IE6, IE7, Opera, Safari).
          Hide
          Jochen Wiedmann added a comment -

          Applied, thank you!

          Show
          Jochen Wiedmann added a comment - Applied, thank you!
          Hide
          Rob Slifka added a comment -

          Many thanks Jochen! I see that another has verified the fix in FILEUPLOAD-138.

          Any idea when we'll see 1.2.1 released?

          Show
          Rob Slifka added a comment - Many thanks Jochen! I see that another has verified the fix in FILEUPLOAD-138 . Any idea when we'll see 1.2.1 released?
          Hide
          Clint Popetz added a comment -

          I'd like to note that this is actually browser independent and not just an issue with short files...when data is coming via AJP (coyote / mod_jk) in tomcat, it comes in chunks, and if a chunk falls on a boundary, it triggers this bug (which it does for my web site a lot and I've been chasing it for weeks) because the first read will just return what's left in that chunk (not enough for a boundary) and the second read would return the rest of the boundary.

          Show
          Clint Popetz added a comment - I'd like to note that this is actually browser independent and not just an issue with short files...when data is coming via AJP (coyote / mod_jk) in tomcat, it comes in chunks, and if a chunk falls on a boundary, it triggers this bug (which it does for my web site a lot and I've been chasing it for weeks) because the first read will just return what's left in that chunk (not enough for a boundary) and the second read would return the rest of the boundary.
          Hide
          Mat Gessel added a comment -

          Incidentally, I started triggering this bug after updating to JRE 1.7 u1 on the client. I have a standalone Java application that is uploading files to a Tomcat container via HTTPS/SSL.

          Reproduction requires:

          • HTTPS/SSL transmission
          • jdk1.7.0_u1 or later
          • commons-fileupload-1.2.0 or previous

          Fixed with commons-fileupload-1.2.1 or later

          Show
          Mat Gessel added a comment - Incidentally, I started triggering this bug after updating to JRE 1.7 u1 on the client . I have a standalone Java application that is uploading files to a Tomcat container via HTTPS/SSL. Reproduction requires: HTTPS/SSL transmission jdk1.7.0_u1 or later commons-fileupload-1.2.0 or previous Fixed with commons-fileupload-1.2.1 or later

            People

            • Assignee:
              Jochen Wiedmann
              Reporter:
              Alexander Sova
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development