Kafka
  1. Kafka
  2. KAFKA-742

Existing directories under the Kafka data directory without any data cause process to not start

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.8.0
    • Fix Version/s: None
    • Component/s: config
    • Labels:
      None

      Description

      I incorrectly setup the configuration file to have the metrics go to /var/kafka/metrics while the logs were in /var/kafka. On startup I received the following error then the daemon exited:

      30 [main] INFO kafka.log.LogManager - [Log Manager on Broker 0] Loading log 'metrics'
      32 [main] FATAL kafka.server.KafkaServerStartable - Fatal error during KafkaServerStable startup. Prepare to shutdown
      java.lang.StringIndexOutOfBoundsException: String index out of range: -1
      at java.lang.String.substring(String.java:1937)
      at kafka.log.LogManager.kafka$log$LogManager$$parseTopicPartitionName(LogManager.scala:335)
      at kafka.log.LogManager$$anonfun$loadLogs$1$$anonfun$apply$3.apply(LogManager.scala:112)
      at kafka.log.LogManager$$anonfun$loadLogs$1$$anonfun$apply$3.apply(LogManager.scala:109)
      at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
      at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34)
      at kafka.log.LogManager$$anonfun$loadLogs$1.apply(LogManager.scala:109)
      at kafka.log.LogManager$$anonfun$loadLogs$1.apply(LogManager.scala:101)
      at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
      at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:32)
      at kafka.log.LogManager.loadLogs(LogManager.scala:101)
      at kafka.log.LogManager.<init>(LogManager.scala:62)
      at kafka.server.KafkaServer.startup(KafkaServer.scala:59)
      at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:34)
      at kafka.Kafka$.main(Kafka.scala:46)
      at kafka.Kafka.main(Kafka.scala)
      34 [main] INFO kafka.server.KafkaServer - [Kafka Server 0], shutting down

      This was on a brand new cluster so no data or metrics logs existed yet.

      Moving the metrics to their own directory (not a child of the logs) allowed the daemon to start.

      Took a few minutes to figure out what was wrong.

        Activity

        Hide
        Chris Curtin added a comment -

        Thanks Jay, a better error is a good idea for this.

        Show
        Chris Curtin added a comment - Thanks Jay, a better error is a good idea for this.
        Hide
        Jay Kreps added a comment -

        Getting an error is not really bad since we assume we have control of everything under the log directory (it is obviously hard for us to distinguish a log directory from a non-log directory and silently ignoring could be worse than an error). So I think there are two problems here:
        1. Are we enabling metrics logging by default? Are we creating the metrics dir even if metrics logging is not enabled. This needs to be sanity checked...
        2. If there are bogus directories under the log directory I think the right thing to do is to give a better error message (something like "Found directory /x/y/z, 'z' is not in the form topic-partition").

        If you agree with those fixes I will take this on.

        Show
        Jay Kreps added a comment - Getting an error is not really bad since we assume we have control of everything under the log directory (it is obviously hard for us to distinguish a log directory from a non-log directory and silently ignoring could be worse than an error). So I think there are two problems here: 1. Are we enabling metrics logging by default? Are we creating the metrics dir even if metrics logging is not enabled. This needs to be sanity checked... 2. If there are bogus directories under the log directory I think the right thing to do is to give a better error message (something like "Found directory /x/y/z, 'z' is not in the form topic-partition"). If you agree with those fixes I will take this on.
        Chris Curtin created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Chris Curtin
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development