Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-21075

Kafka broker goes down after Ambari upgrade from 2.5.0 to 2.5.1 due to missing 'kafka.timeline.metrics.instanceId' property

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Duplicate
    • Affects Version/s: 2.5.1
    • Fix Version/s: 2.5.1
    • Component/s: ambari-upgrade
    • Labels:
      None

      Description

      Kafka logs indicate:

      [2017-05-19 03:42:17,105] FATAL  (kafka.Kafka$)
      java.lang.IllegalArgumentException: requirement failed: Missing required property 'kafka.timeline.metrics.instanceId'
              at scala.Predef$.require(Predef.scala:233)
              at kafka.utils.VerifiableProperties.getString(VerifiableProperties.scala:177)
              at org.apache.hadoop.metrics2.sink.kafka.KafkaTimelineMetricsReporter.init(KafkaTimelineMetricsReporter.java:169)
              at kafka.metrics.KafkaMetricsReporter$$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:66)
              at kafka.metrics.KafkaMetricsReporter$$anonfun$startReporters$1.apply(KafkaMetricsReporter.scala:64)
              at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
              at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:34)
              at kafka.metrics.KafkaMetricsReporter$.startReporters(KafkaMetricsReporter.scala:64)
              at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:27)
              at kafka.Kafka$.main(Kafka.scala:58)
              at kafka.Kafka.main(Kafka.scala)
      

      Did not see this property being added on ambari_upgrade_config_changes.json file
      On a different upgrade path like fromAmbari-2.2.2.o to 2.5.1, this property got added

      [root@ ambari-server]# cat ambari_upgrade_config_changes_05-19-2017_00\:23\:28.json | grep -C4 kafka.timeline.metrics.instanceId
          "kafka-broker": {
            "Added": {
              "kafka.timeline.metrics.hosts": "{{ams_collector_hosts}}",
              "kafka.timeline.metrics.set.instanceId": "{{set_instanceId}}",
              "kafka.timeline.metrics.instanceId": "{{cluster_name}}",
              "port": "6667"
            },
      

        Attachments

        1. AMBARI-21075-trunk.patch
          2 kB
          Dmytro Sen

          Issue Links

            Activity

              People

              • Assignee:
                dsen Dmytro Sen
                Reporter:
                dsen Dmytro Sen
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: