Uploaded image for project: 'Camel'
  1. Camel
  2. CAMEL-12211

Idempotent on camel 2.20.0 and 2.20.1 allows duplication

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 2.20.0, 2.20.1
    • Fix Version/s: 2.21.0
    • Component/s: camel-core
    • Labels:
      None
    • Environment:

      Reproduced on Linux and OSX

    • Estimated Complexity:
      Unknown

      Description

      On a project with at least two routes and idempotent repository on both source endpoints and another on the route to avoid duplication when the number of exchanges goes beyond certain size the target starts to get duplicates. 

      Have created a sample project where the problem can be reproduced on https://github.com/pampanz/camel-idempotent-issue

      Details on how to use the project and reproduce the issue are on the README.md of the project.

       

      The route setup in the project is

      from("file://./fromSource?delay=5000&idempotent=true&idempotentKey=\$simple

      {[file:name|file:///name]}

      &idempotentRepository=#myRepo")
      .autoStartup(true)
      .routeId("source")
      .log("Received file [\${file:name}], proceed to move to target")
      .to("direct:dedup")

      from("file://./fromSource2?delay=5000&idempotent=true&idempotentKey=\$simple

      {[file:name|file:///name]}

      &idempotentRepository=#myRepo")
      .autoStartup(true)
      .routeId("source2")
      .log("Received file [\${file:name}], proceed to move to target")
      .to("direct:dedup")

      from("direct:dedup")
      .routeId("Deduping and moving")
      .log("Deduping file [\${file:name}]")
      .idempotentConsumer(simple('${file:name}'), FileIdempotentRepository.fileIdempotentRepository(new File("dedup.idempotent.file.store"), 100000, 1000000))
      .setHeader("CamelFileName",simple('${file:name}-${date:now:yyyyMMddhhmmss}'))
      .log("Final file to the target is [\${file:name}]")
      .to("file://./toTarget")

       

      If more than a thousand files are available on both sources once the server finish processing them the same files are copied over then camel 2.20.0 and 2.20.1 starts creating duplicates on the target folder. The same DOES NOT happen on 1.19.4, 1.19.0. We haven't tried but are pretty sure it didn't happen on previous versions either.

        Attachments

          Activity

            People

            • Assignee:
              davsclaus Claus Ibsen
              Reporter:
              pampanz Pablo Sosa
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: