Kafka
  1. Kafka
  2. KAFKA-682

java.lang.OutOfMemoryError: Java heap space

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.8.0
    • Fix Version/s: None
    • Component/s: core
    • Labels:
      None
    • Environment:

      Description

      git pull (commit 32dae955d5e2e2dd45bddb628cb07c874241d856)

      ...build...

      ./sbt update
      ./sbt package

      ...run...

      bin/zookeeper-server-start.sh config/zookeeper.properties
      bin/kafka-server-start.sh config/server.properties

      ...then configured fluentd with kafka plugin...

      gem install fluentd --no-ri --no-rdoc
      gem install fluent-plugin-kafka
      fluentd -c ./fluent/fluent.conf -vv

      ...then flood fluentd with messages inputted from syslog and outputted to kafka.

      results in (after about 10000 messages of 1K each in 3s):

      [2013-01-05 02:00:52,087] ERROR Closing socket for /127.0.0.1 because of error (kafka.network.Processor)
      java.lang.OutOfMemoryError: Java heap space
      at kafka.api.ProducerRequest$$anonfun$1$$anonfun$apply$1.apply(ProducerRequest.scala:45)
      at kafka.api.ProducerRequest$$anonfun$1$$anonfun$apply$1.apply(ProducerRequest.scala:42)
      at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
      at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:206)
      at scala.collection.immutable.Range$ByOne$class.foreach(Range.scala:282)
      at scala.collection.immutable.Range$$anon$1.foreach(Range.scala:274)
      at scala.collection.TraversableLike$class.map(TraversableLike.scala:206)
      at scala.collection.immutable.Range.map(Range.scala:39)
      at kafka.api.ProducerRequest$$anonfun$1.apply(ProducerRequest.scala:42)
      at kafka.api.ProducerRequest$$anonfun$1.apply(ProducerRequest.scala:38)
      at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:227)
      at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:227)
      at scala.collection.immutable.Range$ByOne$class.foreach(Range.scala:282)
      at scala.collection.immutable.Range$$anon$1.foreach(Range.scala:274)
      at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:227)
      at scala.collection.immutable.Range.flatMap(Range.scala:39)
      at kafka.api.ProducerRequest$.readFrom(ProducerRequest.scala:38)
      at kafka.api.RequestKeys$$anonfun$1.apply(RequestKeys.scala:32)
      at kafka.api.RequestKeys$$anonfun$1.apply(RequestKeys.scala:32)
      at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:47)
      at kafka.network.Processor.read(SocketServer.scala:298)
      at kafka.network.Processor.run(SocketServer.scala:209)
      at java.lang.Thread.run(Thread.java:722)

      1. java_pid22281_Leak_Suspects.zip
        65 kB
        Ricky Ng-Adam
      2. java_pid22281.hprof.gz
        2.93 MB
        Ricky Ng-Adam

        Activity

        Ricky Ng-Adam created issue -
        Ricky Ng-Adam made changes -
        Field Original Value New Value
        Attachment java_pid22281.hprof.gz [ 12563466 ]
        Attachment java_pid22281_Leak_Suspects.zip [ 12563467 ]
        Jay Kreps made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Ricky Ng-Adam
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development