Qpid
  1. Qpid
  2. QPID-2902

LargeMessageTest fails on java.0.10 test profiles

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9
    • Component/s: Java Broker, Java Client
    • Labels:
      None

      Description

      LargeMessageTest sporadically fails on the java.0.10 test profiles. It is currently excluded.

      eg:
      FAILED
      Excpetion occured:java.nio.charset.MalformedInputException: Input length = 1
      junit.framework.AssertionFailedError: Excpetion occured:java.nio.charset.MalformedInputException: Input length = 1
      at org.apache.qpid.test.unit.basic.LargeMessageTest.checkLargeMessage(LargeMessageTest.java:159)
      at org.apache.qpid.test.unit.basic.LargeMessageTest.test1024k(LargeMessageTest.java:133)
      at org.apache.qpid.test.utils.QpidBrokerTestCase.runBare(QpidBrokerTestCase.java:232)
      at org.apache.qpid.test.utils.QpidTestCase.run(QpidTestCase.java:120)

      1. QPID-2902.patch
        0.7 kB
        Danushka Menikkumbura

        Activity

        Hide
        Robbie Gemmell added a comment -

        Updating 'Fix For' to Unknown on issues not targeted for 0.8

        Show
        Robbie Gemmell added a comment - Updating 'Fix For' to Unknown on issues not targeted for 0.8
        Hide
        Danushka Menikkumbura added a comment -

        Initially it looked as if it was an encoding issue but when I had a closer look I could see that the issue lied in the frame assembler.

        Actually the message segments generated in the Assembler.java (lines 158-162) were corrupted as a result of scrambled frame body content.

        Assembler.frame() receives frames of a given message that are stored in a list and assembled and sent forward as soon as the last frame is received. The frame body content gets corrupted during this transition. This is due to a flaw in the frame body content formation logic in the broker transport input handler.

        The InputHandler uses a fixed-length ByteBuffer which gets filled with incoming data stream and frame content is formed by slicing this buffer. When a ByteBuffer is sliced, the new buffer is a shared subsequence of the original buffer's content and changes to the original buffer is visible in the new buffer and vice versa. Because of this reason, frame content stored in the segment gets corrupted and as a result the content of final AMQP message gets corrupted.

        Thanks,
        Danushka

        Show
        Danushka Menikkumbura added a comment - Initially it looked as if it was an encoding issue but when I had a closer look I could see that the issue lied in the frame assembler. Actually the message segments generated in the Assembler.java (lines 158-162) were corrupted as a result of scrambled frame body content. Assembler.frame() receives frames of a given message that are stored in a list and assembled and sent forward as soon as the last frame is received. The frame body content gets corrupted during this transition. This is due to a flaw in the frame body content formation logic in the broker transport input handler. The InputHandler uses a fixed-length ByteBuffer which gets filled with incoming data stream and frame content is formed by slicing this buffer. When a ByteBuffer is sliced, the new buffer is a shared subsequence of the original buffer's content and changes to the original buffer is visible in the new buffer and vice versa. Because of this reason, frame content stored in the segment gets corrupted and as a result the content of final AMQP message gets corrupted. Thanks, Danushka
        Hide
        Danushka Menikkumbura added a comment -

        Please patch the InputHandler.java file in transport/network using the attached patch file (QPID-2902.patch).

        Thanks,
        Danushka

        Show
        Danushka Menikkumbura added a comment - Please patch the InputHandler.java file in transport/network using the attached patch file ( QPID-2902 .patch). Thanks, Danushka
        Hide
        Robbie Gemmell added a comment - - edited

        Whilst investigating another issue I came across QPID-3010, and after applying the fix for that LargeMessageTest hasn't failed since.

        The issue here was not strictly the InputHandler slicing the ByteBuffer, as only the Java broker was affected; the Java client was known to work ok against the C++ broker despite use of exactly the same classes suggesting the underlying cause actually lay elsewhere. The InputHandler is deliberately not copying the ByteBuffer to prevent the expense of doing so, since the Assembler is about to copy the frames into the completed command buffer before passing it on to the connection level. The root cause of the problem was that MINA was not configured in the expected way on the broker and so was free to reuse the ByteBuffers at any point following exit from the InputHandler.received() method due to use of its PooledByteBufferAllocator. It was also free to make use of DirectByteBuffers instead of HeapByteBuffers and this was the cause of the issue I started investigating.

        As this problem was actually caused by another bug and avoiding the expense of copying the ByteBuffers on both the client (which worked fine) and broker seems desirable, I don't believe the attached patch should or needs to be applied.

        Show
        Robbie Gemmell added a comment - - edited Whilst investigating another issue I came across QPID-3010 , and after applying the fix for that LargeMessageTest hasn't failed since. The issue here was not strictly the InputHandler slicing the ByteBuffer, as only the Java broker was affected; the Java client was known to work ok against the C++ broker despite use of exactly the same classes suggesting the underlying cause actually lay elsewhere. The InputHandler is deliberately not copying the ByteBuffer to prevent the expense of doing so, since the Assembler is about to copy the frames into the completed command buffer before passing it on to the connection level. The root cause of the problem was that MINA was not configured in the expected way on the broker and so was free to reuse the ByteBuffers at any point following exit from the InputHandler.received() method due to use of its PooledByteBufferAllocator. It was also free to make use of DirectByteBuffers instead of HeapByteBuffers and this was the cause of the issue I started investigating. As this problem was actually caused by another bug and avoiding the expense of copying the ByteBuffers on both the client (which worked fine) and broker seems desirable, I don't believe the attached patch should or needs to be applied.
        Hide
        Robbie Gemmell added a comment -

        Andrew can you review this please? Thanks.

        Show
        Robbie Gemmell added a comment - Andrew can you review this please? Thanks.

          People

          • Assignee:
            Andrew Kennedy
            Reporter:
            Robbie Gemmell
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development