[CAMEL-5582] PGP data format doesn't close file input streams

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.10.1
• Fix Version/s: 2.9.4, 2.10.2, 2.11.0
• Component/s: camel-crypto

Environment

Windows XP

• Estimated Complexity: Moderate
• Assignee: Claus Ibsen
• Reporter: Daniel Gredler
• Created: 07/Sep/12 21:16
• Updated: 11/Sep/12 06:51
• Resolved: 09/Sep/12 08:24

Description

The following route doesn't work on Windows:

from("file:foo").marshal().pgp(...).to("ftp:blah")

The reason is that PGPDataFormat.marshal() and PGPDataFormat.unmarshal() both use IOUtils.toByteArray() to read the incoming stream into a byte array, but neither of these two methods closes the streams after they have been read from. My understanding is that not closing these streams prevents Camel from subsequently renaming the file once the route has completed. I've attached a sample stack trace at the bottom.

The following seems to fix the issue for me:

Replacing the following code in org.apache.camel.converter.crypto.PGPDataFormat.marshal(...):

InputStream plaintextStream = ExchangeHelper.convertToMandatoryType(exchange, InputStream.class, graph);

byte[] compressedData = PGPDataFormatUtil.compress(IOUtils.toByteArray(plaintextStream),
        PGPLiteralData.CONSOLE, CompressionAlgorithmTags.ZIP);

With this code:

byte[] plaintextData;
InputStream plaintextStream = null;
try {
    plaintextStream = ExchangeHelper.convertToMandatoryType(exchange, InputStream.class, graph);
    plaintextData = IOUtils.toByteArray(plaintextStream);
} finally {
    IOUtils.closeQuietly(plaintextStream);
}

byte[] compressedData = PGPDataFormatUtil.compress(plaintextData, PGPLiteralData.CONSOLE, CompressionAlgorithmTags.ZIP);

And replacing the following code in org.apache.camel.converter.crypto.PGPDataFormat.unmarshal(...):

InputStream in = new ByteArrayInputStream(IOUtils.toByteArray(encryptedStream));
in = PGPUtil.getDecoderStream(in);

With this code:

InputStream in;
try {
    byte[] encryptedData = IOUtils.toByteArray(encryptedStream);
    InputStream byteStream = new ByteArrayInputStream(encryptedData);
    in = PGPUtil.getDecoderStream(byteStream);
} finally {
    IOUtils.closeQuietly(encryptedStream);
}

And here's the stack trace mentioned above:

org.apache.camel.component.file.GenericFileOperationFailedException: Error renaming file from C:\opt\connect\just\a\test\lax\blah.txt to C:\opt\connect\just\a\test\lax\.sent\blah.txt
    at org.apache.camel.component.file.FileOperations.renameFile(FileOperations.java:72)
    at org.apache.camel.component.file.strategy.GenericFileProcessStrategySupport.renameFile(GenericFileProcessStrategySupport.java:107)
    at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:86)
    at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:132)
    at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:82)
    at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:53)
    at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:55)
    at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:226)
    at org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:199)
    at org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:37)
    at org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:157)
    at org.apache.camel.processor.RouteContextProcessor$1.done(RouteContextProcessor.java:56)
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:106)
    at org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:45)
    at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
    at org.apache.camel.processor.UnitOfWorkProcessor.processAsync(UnitOfWorkProcessor.java:150)
    at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:117)
    at org.apache.camel.processor.RouteInflightRepositoryProcessor.processNext(RouteInflightRepositoryProcessor.java:48)
    at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
    at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)
    at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)
    at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)
    at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:73)
    at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:336)
    at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:189)
    at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:155)
    at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:139)
    at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:91)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
    at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
    at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: Renaming file from: C:\opt\connect\just\a\test\lax\blah.txt to: C:\opt\connect\just\a\test\lax\.sent\blah.txt failed due cannot delete from file: C:\opt\connect\just\a\test\lax\blah.txt after copy succeeded
    at org.apache.camel.util.FileUtil.renameFile(FileUtil.java:362)
    at org.apache.camel.component.file.FileOperations.renameFile(FileOperations.java:70)
    ... 36 more