Camel
  1. Camel
  2. CAMEL-4686

Messages larger than 64KB can be processed very slowly due to file read buffer of 1 byte

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.7.5, 2.8.3, 2.9.0
    • Component/s: camel-core
    • Labels:
      None
    • Environment:

      ServiceMix 4.3, Solaris 10

    • Patch Info:
      Patch Available
    • Estimated Complexity:
      Novice

      Description

      When the message body is larger than 64KB, the message is stored on the disk by CachedOutputStream and read back later by FileInputStreamCache.
      A thread dump during a load test shows that the execution is most of the time waiting in the "InputStream.read()" method calls.
      The "truss" output shows that the file is read back 1 byte at a time, which makes the calls very slow.

      A first workaround (setting property "CamelCachedOutputStreamThreshold" to 10000000 on the CamelContext's properties) helped in some case.

      The cause is that the FileInputStreamCache does not use a BufferInputStream to read on the disk, although the
      CachedOutputStream does properly use a BufferOutputStream.
      See
      http://java.sun.com/docs/books/performance/1st_edition/html/JPIOPerformance.fm.html#17675

      A trivial patch is attached, which provides a fix.

      Reference :
      http://servicemix.396122.n5.nabble.com/DefaultTraceFormatter-is-very-slow-for-messages-gt-64KB-td4902840.html
      This problem was first noticed when the trace was enabled on the CamelContext, but it occurs also in other scenarios.

        Issue Links

          Activity

          metatech created issue -
          metatech made changes -
          Field Original Value New Value
          Attachment FileInputStreamCache_buffered.patch [ 12503870 ]
          metatech made changes -
          Description When the message body is larger than 64KB, the message is stored on the disk by CachedOutputStream and read back later by FileInputStreamCache.
          A thread dump during a load test shows that the execution is most of the time waiting in the "InputStream.read()" method calls.
          The "truss" output shows that the file is read back 1 byte at a time, which makes the calls very slow.

          A first workaround (setting property "CamelCachedOutputStreamThreshold" to 10000000 on the CamelContext's properties) helped in some case.

          The cause is that the FileInputStreamCache does not use a BufferInputStream to read on the disk, although the CachedOutputStream does properly use a BufferOutputStream.

          A trivial patch is attached, which provides a fix.

          Reference :
          http://servicemix.396122.n5.nabble.com/DefaultTraceFormatter-is-very-slow-for-messages-gt-64KB-td4902840.html
          This problem was first noticed when the trace was enabled on the CamelContext, but it occurs also in other scenarios.
          When the message body is larger than 64KB, the message is stored on the disk by CachedOutputStream and read back later by FileInputStreamCache.
          A thread dump during a load test shows that the execution is most of the time waiting in the "InputStream.read()" method calls.
          The "truss" output shows that the file is read back 1 byte at a time, which makes the calls very slow.

          A first workaround (setting property "CamelCachedOutputStreamThreshold" to 10000000 on the CamelContext's properties) helped in some case.

          The cause is that the FileInputStreamCache does not use a BufferInputStream to read on the disk, although the
          CachedOutputStream does properly use a BufferOutputStream.
          See
          http://java.sun.com/docs/books/performance/1st_edition/html/JPIOPerformance.fm.html#17675

          A trivial patch is attached, which provides a fix.

          Reference :
          http://servicemix.396122.n5.nabble.com/DefaultTraceFormatter-is-very-slow-for-messages-gt-64KB-td4902840.html
          This problem was first noticed when the trace was enabled on the CamelContext, but it occurs also in other scenarios.
          Claus Ibsen made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Assignee Claus Ibsen [ davsclaus ]
          Fix Version/s 2.7.5 [ 12318648 ]
          Fix Version/s 2.8.3 [ 12318649 ]
          Fix Version/s 2.9.0 [ 12316374 ]
          Resolution Fixed [ 1 ]
          Claus Ibsen made changes -
          Issue Type Bug [ 1 ] Improvement [ 4 ]
          Claus Ibsen made changes -
          Link This issue relates to CAMEL-4687 [ CAMEL-4687 ]

            People

            • Assignee:
              Claus Ibsen
              Reporter:
              metatech
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development