Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3134

Missing required configuration "value.deserializer" when initializing a KafkaConsumer with a valid "valueDeserializer"

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.9.0.0
    • Fix Version/s: 0.9.0.1, 0.10.0.0
    • Component/s: None
    • Labels:
      None

      Description

      I tried to initialize a KafkaConsumer object using with a null keyDeserializer and a non-null valueDeserializer:

      public KafkaConsumer(Properties properties, Deserializer<K> keyDeserializer,
                               Deserializer<V> valueDeserializer)
      

      Then I got an exception as follows:

      Caused by: org.apache.kafka.common.config.ConfigException: Missing required configuration "value.deserializer" which has no default value.
      	at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:148)
      	at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:49)
      	at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:56)
      	at org.apache.kafka.clients.consumer.ConsumerConfig.<init>(ConsumerConfig.java:336)
      	at org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:518)
      	.....
      

      Then I went to ConsumerConfig.java file and found this block of code causing the problem:

      public static Map<String, Object> addDeserializerToConfig(Map<String, Object> configs,
                                                                    Deserializer<?> keyDeserializer,
                                                                    Deserializer<?> valueDeserializer) {
              Map<String, Object> newConfigs = new HashMap<String, Object>();
              newConfigs.putAll(configs);
              if (keyDeserializer != null)
                  newConfigs.put(KEY_DESERIALIZER_CLASS_CONFIG, keyDeserializer.getClass());
              if (keyDeserializer != null)
                  newConfigs.put(VALUE_DESERIALIZER_CLASS_CONFIG, valueDeserializer.getClass());
              return newConfigs;
          }
      
          public static Properties addDeserializerToConfig(Properties properties,
                                                           Deserializer<?> keyDeserializer,
                                                           Deserializer<?> valueDeserializer) {
              Properties newProperties = new Properties();
              newProperties.putAll(properties);
              if (keyDeserializer != null)
                  newProperties.put(KEY_DESERIALIZER_CLASS_CONFIG, keyDeserializer.getClass().getName());
              if (keyDeserializer != null)
                  newProperties.put(VALUE_DESERIALIZER_CLASS_CONFIG, valueDeserializer.getClass().getName());
              return newProperties;
          }
      

      Instead of checking valueDeserializer, the code checks keyDeserializer every time. So when keyDeserializer is null but valueDeserializer is not, the valueDeserializer property will never get set.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              yying1 Yifan Ying
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: