Uploaded image for project: 'CXF'
  1. CXF
  2. CXF-8070

CXF error while signing an attachment over 2GB size

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 3.2.6
    • Fix Version/s: None
    • Component/s: WS-* Components
    • Labels:
      None
    • Estimated Complexity:
      Unknown

      Description

      Hello,

      We are using Apache CXF 3.2.6 and WSS4J 2.2.2 as WS-Security implementation in our project in order to sign, encrypt and send SOAP messages.

      The current implementation is 'SOAP with Attachments/MIME parts' and we cannot switch to MTOM.

      1. We have tried to sign, encrypt and send an attachment larger than 2 GB but we received the following error:

      2019-06-27 16:30:59,322 [] [default] [0c33b5dc-be5e-406a-97ca-1dcceee980be@domibus.eu] ERROR e.d.e.s.UserMessageSender:162 - Error sending message [0c33b5dc-be5e-406a-97ca-1dcceee980be@domibus.eu]
      java.lang.OutOfMemoryError: Required array size too large
          at java.io.BufferedInputStream.fill(BufferedInputStream.java:227)
          at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
          at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
          at java.io.FilterInputStream.read(FilterInputStream.java:107)
          at org.apache.xml.security.stax.ext.XMLSecurityUtils.copy(XMLSecurityUtils.java:463)
          at org.apache.xml.security.stax.impl.transformer.TransformIdentity.transform(TransformIdentity.java:184)
          at org.apache.wss4j.stax.impl.transformer.AttachmentContentSignatureTransform.transform(AttachmentContentSignatureTransform.java:110)
          at org.apache.wss4j.stax.impl.processor.output.WSSSignatureOutputProcessor.digestExternalReference(WSSSignatureOutputProcessor.java:211)
          at org.apache.xml.security.stax.impl.processor.output.AbstractSignatureOutputProcessor.doFinalInternal(AbstractSignatureOutputProcessor.java:82)
          at org.apache.wss4j.stax.impl.processor.output.WSSSignatureOutputProcessor.processEvent(WSSSignatureOutputProcessor.java:138)
          at org.apache.xml.security.stax.ext.AbstractOutputProcessor.processNextEvent(AbstractOutputProcessor.java:133)
          at org.apache.xml.security.stax.impl.OutputProcessorChainImpl.processEvent(OutputProcessorChainImpl.java:212)
          at org.apache.xml.security.stax.impl.XMLSecurityStreamWriter.chainProcessEvent(XMLSecurityStreamWriter.java:62)
          at org.apache.xml.security.stax.impl.XMLSecurityStreamWriter.outputOpenStartElement(XMLSecurityStreamWriter.java:83)
          at org.apache.xml.security.stax.impl.XMLSecurityStreamWriter.writeEndElement(XMLSecurityStreamWriter.java:164)
          at org.apache.cxf.staxutils.StaxUtils.copy(StaxUtils.java:762)
          at org.apache.cxf.staxutils.StaxUtils.copy(StaxUtils.java:722)
          at org.apache.cxf.binding.soap.saaj.SAAJOutInterceptor$SAAJOutEndingInterceptor.handleMessage(SAAJOutInterceptor.java:214)
          at org.apache.cxf.binding.soap.saaj.SAAJOutInterceptor$SAAJOutEndingInterceptor.handleMessage(SAAJOutInterceptor.java:174)
          at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308)
          at org.apache.cxf.endpoint.ClientImpl.doInvoke(ClientImpl.java:537)
          at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:446)
          at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:361)
          at org.apache.cxf.endpoint.ClientImpl.invoke(ClientImpl.java:319)
          at org.apache.cxf.endpoint.ClientImpl.invokeWrapped(ClientImpl.java:354)
          at org.apache.cxf.jaxws.DispatchImpl.invoke(DispatchImpl.java:322)
          at org.apache.cxf.jaxws.DispatchImpl.invoke(DispatchImpl.java:241)
          at eu.domibus.ebms3.sender.MSHDispatcher.dispatch(MSHDispatcher.java:61)
          at eu.domibus.ebms3.sender.MSHDispatcher$$FastClassBySpringCGLIB$$105974a1.invoke(<generated>)
          at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
          at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
          at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
          at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
          at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
          at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
          at eu.domibus.ebms3.sender.MSHDispatcher$$EnhancerBySpringCGLIB$$d1cb6add.dispatch(<generated>)
          at eu.domibus.ebms3.sender.AbstractUserMessageSender.sendMessage(AbstractUserMessageSender.java:140)
          at eu.domibus.ebms3.sender.AbstractUserMessageSender$$FastClassBySpringCGLIB$$9b7953e7.invoke(<generated>)
          at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
          at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
          at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
          at eu.domibus.common.metrics.MetricsAspect.surroundWithATimer(MetricsAspect.java:38)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627)
          at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616)
          at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168)
          at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
          at eu.domibus.common.metrics.MetricsAspect.surroundWithACounter(MetricsAspect.java:54)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627)
          at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616)
          at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168)
          at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
          at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
          at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168)
          at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
          at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
      
      

      2. Modifying an existing JUnit from WSS4J sources (org.apache.wss4j.stax.test.AttachmentTest) to sign an attachment over 2GB we are getting quite the same error:

      Connected to the target VM, address: '127.0.0.1:10317', transport: 'socket'
      Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
          at java.util.Arrays.copyOf(Arrays.java:3332)
          at java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:124)
          at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:448)
          at java.lang.StringBuilder.append(StringBuilder.java:136)
          at org.apache.xml.security.stax.impl.util.DigestOutputStream.write(DigestOutputStream.java:64)
          at org.apache.wss4j.common.util.CRLFOutputStream.write(CRLFOutputStream.java:71)
          at org.apache.xml.security.stax.ext.XMLSecurityUtils.copy(XMLSecurityUtils.java:475)
          at org.apache.xml.security.stax.impl.transformer.TransformIdentity.transform(TransformIdentity.java:179)
          at org.apache.wss4j.stax.impl.transformer.AttachmentContentSignatureTransform.transform(AttachmentContentSignatureTransform.java:108)
          at org.apache.wss4j.stax.impl.transformer.AttachmentCompleteSignatureTransform.transform(AttachmentCompleteSignatureTransform.java:51)
          at org.apache.wss4j.stax.impl.processor.output.WSSSignatureOutputProcessor.digestExternalReference(WSSSignatureOutputProcessor.java:212)
          at org.apache.xml.security.stax.impl.processor.output.AbstractSignatureOutputProcessor.doFinalInternal(AbstractSignatureOutputProcessor.java:95)
          at org.apache.wss4j.stax.impl.processor.output.WSSSignatureOutputProcessor.processEvent(WSSSignatureOutputProcessor.java:139)
          at org.apache.xml.security.stax.ext.AbstractOutputProcessor.processNextEvent(AbstractOutputProcessor.java:133)
          at org.apache.xml.security.stax.impl.OutputProcessorChainImpl.processEvent(OutputProcessorChainImpl.java:217)
          at org.apache.xml.security.stax.impl.XMLSecurityStreamWriter.chainProcessEvent(XMLSecurityStreamWriter.java:62)
          at org.apache.xml.security.stax.impl.XMLSecurityStreamWriter.outputOpenStartElement(XMLSecurityStreamWriter.java:83)
          at org.apache.xml.security.stax.impl.XMLSecurityStreamWriter.writeCharacters(XMLSecurityStreamWriter.java:302)
          at org.apache.wss4j.stax.test.utils.XmlReaderToWriter.write(XmlReaderToWriter.java:88)
          at org.apache.wss4j.stax.test.utils.XmlReaderToWriter.writeAll(XmlReaderToWriter.java:36)
          at org.apache.wss4j.stax.test.AttachmentTest.testMultipleAttachmentCompleteSignature_Catalin(AttachmentTest.java:474)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:628)
          at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:117)
          at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:184)
          at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor$$Lambda$208/651802632.execute(Unknown Source)
          at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
          at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:180)
          at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:127)
      
      

      3. Then checking the source code, we saw that there is a hardcoded limitation to 2GB:

      WSSSignatureOutputProcessor class from wss4j-ws-security-stax.jar which is used for sign the attachments has a hardcoded limit of 2GB to accept:

      line 198:

                          DigestOutputStream digestOutputStream = createMessageDigestOutputStream(signaturePartDef.getDigestAlgo());
                          InputStream inputStream = attachment.getSourceStream();
                          if (!inputStream.markSupported()) {
                              inputStream = new BufferedInputStream(inputStream);
                          }
                          inputStream.mark(Integer.MAX_VALUE); //we can process at maximum 2G with the standard jdk streams
      

      Question is: there is a fix/solution in order to send more than 2GB for a single attachment?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              cenache Catalin Enache
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: