[CAMEL-20035] Program terminates with OutOfMemoryError

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 3.20.9, 3.21.3, 3.22.0, 4.0.3, 4.2.0
• Component/s: camel-core
• Estimated Complexity: Unknown
• Assignee: Claus Ibsen
• Reporter: Marco Bungart
• Created: 23/Oct/23 18:47
• Updated: 25/Oct/23 07:27
• Resolved: 25/Oct/23 07:27

Description

Given the following route:

public class Timer {
  ...
  static RouteBuilder timerRoute() {
    return new RouteBuilder() {
      private final AtomicInteger calls = new AtomicInteger();

      @Override
      public void configure() {
        // @formatter:off
        from(
            timer("timer")
                .period(Duration.ofMillis(100).toMillis())
                .fixedRate(true))
            .process(exchange -> {
              log.info("calls: {}", calls.incrementAndGet());
              exchange.getIn().setBody(new LargeObject());
            })
            .log("${body.fieldTwo}");
        // @formatter:on
      }
    };
  }
}

and the following implementation of LargeObject

public class LargeObject {
  private static final Random RANDOM = new Random();

  byte[] data;
  String fieldOne;
  String fieldTwo;

  public LargeObject() {
    this.data = new byte[100 * 1024 * 1024]; // 100 MB
    RANDOM.nextBytes(data);
    fieldOne = UUID.randomUUID().toString();
    fieldTwo = UUID.randomUUID().toString();
  }
}

We would expect that the program runs indefinitely. But if we limit the memory to e.g. ~3.2 GB (which is 10% of the whole system memory in my case) by running:

java \
  -XX:MinRAMPercentage=10 \
  -XX:MaxRAMPercentage=10 \
  -XX:+ExitOnOutOfMemoryError \
  -jar ...

The program terminates with an OutOfMemoryError in the 32nd execution of the route.

—

Reproducer 1 (pure camel, github.com)
Reproducer 2 (quarkus camel, allow to define the size of the LargeObject, github.com)

For both repositories, please read the README.adoc for instructions.

—

Observations:

• For the quarkus-sample and a heap-limitation of ~3.2 GB, the breaking point is an object.size of 2047K/2048K. With 2047K, the program runs indefinitely, with 2048K, the program crashes.
• The problem also exist in native mode, but the breaking point is different
• Looking at the implementation of the SimpleLRUCache (github.com), it seems that this class is not using SoftReference s or WeakReference s. As far as I understand it, this means that if 1000 elements of whatever is stored in this cache exceed the heap limit of the JVM, the program will throw an OutOfMemoryError .
• Conversely, this means that if a program "survives" the first 1000 calls, it will (most likely) run indefinitely.

Issue Links

• relates to: CAMEL-19487 camel-core / camel-bean - Dynamic router instances pass messages to wrong endpoints with @DynamicRouter annotation
• relates to: CAMEL-20038 camel-core - Deprecate LRUWeakCache
• relates to: CAMEL-20039 camel-core - SimpleLRUCache add support for soft cache
• relates to: CAMEL-19999 camel-bean - Allow to configure bean introspection cache on component