Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 0.8.0
    • Fix Version/s: 0.7.2, 0.8.0
    • Component/s: None
    • Labels:

      Description

      Currently, it's hard to figure out which configuration value is being used and whether a new configuration is being picked up. Logging all configuration values during startup time can address this issue. We should cover broker, producer and consumer.

      1. KAFKA-246-all-0.8.patch
        19 kB
        Edward Smith
      2. KAFKA-246-broker-0.8.patch
        8 kB
        Edward Smith

        Activity

        Hide
        Jun Rao added a comment -

        This is a duplicate of KAFKA-181.

        Show
        Jun Rao added a comment - This is a duplicate of KAFKA-181 .
        Hide
        Edward Smith added a comment -

        I refactored my patch and made the changes to the producer and consumer as well. This patch supersede's the broker-0.8.patch file.

        There may need to be list support in Logging.scala, but since I didn't have any familiarity with those settings, I skipped it for now.

        Show
        Edward Smith added a comment - I refactored my patch and made the changes to the producer and consumer as well. This patch supersede's the broker-0.8.patch file. There may need to be list support in Logging.scala, but since I didn't have any familiarity with those settings, I skipped it for now.
        Hide
        Edward Smith added a comment -

        Yes, I agree, it was a little tedious. This is my first foray into scala, and kafka, so kept it simple. I might just round out the producer and consumer the easy way, and let someone who knows what they are doing do the fancy version when they get time. I haven't peeked to see if there is code that iterates the entire config file to find unused values, but, you are right, it would be nice to be warned about a typo.

        Show
        Edward Smith added a comment - Yes, I agree, it was a little tedious. This is my first foray into scala, and kafka, so kept it simple. I might just round out the producer and consumer the easy way, and let someone who knows what they are doing do the fancy version when they get time. I haven't peeked to see if there is code that iterates the entire config file to find unused values, but, you are right, it would be nice to be warned about a typo.
        Hide
        Jay Kreps added a comment -

        Thanks for the patch!

        I wonder if instead of adding a logging statement for each value which is a little repetitive we could instead do the following:
        Create a Props object that wraps Properties and has helpers like getInt, getString, etc. Something like this: https://github.com/voldemort/voldemort/blob/master/src/java/voldemort/utils/Props.java
        Then add something internal to this helper that records whether a given property is used or not (i.e. a property is used if the get method is called for that key). This class could be used to log out all configuration in a single place and also could log WARN messages for any properties that are not used (since that is likely a typo).

        Show
        Jay Kreps added a comment - Thanks for the patch! I wonder if instead of adding a logging statement for each value which is a little repetitive we could instead do the following: Create a Props object that wraps Properties and has helpers like getInt, getString, etc. Something like this: https://github.com/voldemort/voldemort/blob/master/src/java/voldemort/utils/Props.java Then add something internal to this helper that records whether a given property is used or not (i.e. a property is used if the get method is called for that key). This class could be used to log out all configuration in a single place and also could log WARN messages for any properties that are not used (since that is likely a typo).
        Hide
        Edward Smith added a comment -

        Saw label "newbie" and gave it a shot. Attached is a patch for the server, only, in 0.8. If this is acceptable, I'll do the producer and consumer in the same way. I'm confused as to which is more appropriate Logger.getLogger or extends Logging. I see it done both ways in source. Let me know.

        New to scala and kafka and just trying to learn my way around it before incorporating it into a project.

        Show
        Edward Smith added a comment - Saw label "newbie" and gave it a shot. Attached is a patch for the server, only, in 0.8. If this is acceptable, I'll do the producer and consumer in the same way. I'm confused as to which is more appropriate Logger.getLogger or extends Logging. I see it done both ways in source. Let me know. New to scala and kafka and just trying to learn my way around it before incorporating it into a project.

          People

          • Assignee:
            Unassigned
            Reporter:
            Jun Rao
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development