Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      Chukwa has many configuration options, and it's easy to mis-set them. I'd like to contribute a module for doing stronger static checks of configuration at startup.

      In particular, this will flag and log cases where:
      An option is set in the Chukwa config files that is never read by Chukwa.
      A non-numeric value is used for a numeric parameter, non-boolean value for boolean parameter, etc.

      The table of options and types is automatically generated and then human-checked. It should be easy to keep it up to date.
      The changes to the Chukwa codebase should be quite modest: one new Jar, and a handful of lines to invoke that Jar on startup.

      Does this sound like a thing people would like to see?

      1. options.dict
        2 kB
        Ari Rabkin
      2. CHUKWA-527.patch
        7 kB
        Ari Rabkin
      3. confspellcheck.jar
        24 kB
        Ari Rabkin

        Activity

        Hide
        Ari Rabkin added a comment -

        Alright. I am committing this. Let me know if something explodes.

        Show
        Ari Rabkin added a comment - Alright. I am committing this. Let me know if something explodes.
        Hide
        Eric Yang added a comment -

        Looks good so far. Need the following:

        • confspellchecker-LICENSE.txt in lib.
        • A wiki page to add new config to .dict file.

        +1 when the above requirements are met.

        Show
        Eric Yang added a comment - Looks good so far. Need the following: confspellchecker-LICENSE.txt in lib. A wiki page to add new config to .dict file. +1 when the above requirements are met.
        Hide
        Ari Rabkin added a comment -

        See http://code.google.com/p/jchord/source/browse/#svn/trunk/conf_spellchecker for source code and license information for the configuration spellcheck library.

        Show
        Ari Rabkin added a comment - See http://code.google.com/p/jchord/source/browse/#svn/trunk/conf_spellchecker for source code and license information for the configuration spellcheck library.
        Hide
        Ari Rabkin added a comment -

        Includes three things:

        • A configuration spellcheck library, written by me. I intend to release source code to it, but do not wish to integrate the sourcecode into Chukwa.
        • A set of "spellcheck files" specifying existing Chukwa options.
        • Minor modifications to ChukwaAgent, CollectorStub, and ChukwaConfig to invoke the library with the appropriate sets of options.
        Show
        Ari Rabkin added a comment - Includes three things: A configuration spellcheck library, written by me. I intend to release source code to it, but do not wish to integrate the sourcecode into Chukwa. A set of "spellcheck files" specifying existing Chukwa options. Minor modifications to ChukwaAgent, CollectorStub, and ChukwaConfig to invoke the library with the appropriate sets of options.
        Hide
        Ari Rabkin added a comment -

        This is my listing of automatically inferred Chukwa options, along with types. Idea is that if it's mostly auto-generated, it should be easier to keep up to date.

        I can also generate separate dictionaries for each component, if that would be viewed as a significant improvement on one big list.

        Next step is adding the functionality to do the checking.

        Show
        Ari Rabkin added a comment - This is my listing of automatically inferred Chukwa options, along with types. Idea is that if it's mostly auto-generated, it should be easier to keep up to date. I can also generate separate dictionaries for each component, if that would be viewed as a significant improvement on one big list. Next step is adding the functionality to do the checking.
        Hide
        Ari Rabkin added a comment -

        This is my listing of automatically inferred Chukwa options, along with types. Idea is that if it's mostly auto-generated, it should be easier to keep up to date.

        I can also generate separate dictionaries for each component, if that would be viewed as a significant change.

        Next step is adding the functionality to do the checking.

        Show
        Ari Rabkin added a comment - This is my listing of automatically inferred Chukwa options, along with types. Idea is that if it's mostly auto-generated, it should be easier to keep up to date. I can also generate separate dictionaries for each component, if that would be viewed as a significant change. Next step is adding the functionality to do the checking.
        Hide
        Bill Graham added a comment -

        +1. It would be nice to have this output also dump out configs that are used and have this functionality available for each Chukwa component (i.e., Demuxer, Agent, Collector, etc).

        Show
        Bill Graham added a comment - +1. It would be nice to have this output also dump out configs that are used and have this functionality available for each Chukwa component (i.e., Demuxer, Agent, Collector, etc).
        Hide
        Ari Rabkin added a comment -

        The context here is that my ongoing research is on flagging and fixing these kinds of bugs, so I have a tool for it that I want to test. I figured the Chukwa community would be a friendly place to try it, and would benefit from it.

        Show
        Ari Rabkin added a comment - The context here is that my ongoing research is on flagging and fixing these kinds of bugs, so I have a tool for it that I want to test. I figured the Chukwa community would be a friendly place to try it, and would benefit from it.

          People

          • Assignee:
            Ari Rabkin
            Reporter:
            Ari Rabkin
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development