Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-490

Allow mdl.xml to be split into multiple files for ease of maintenance

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.4.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      We have nearly two dozen different demux implementations that generate data that is eventually saved into MySQL. On top of this, each different data type has about twenty or more columns in its resulting table. Specifying all of these in a single mdl.xml file has caused us some headaches, so we looked into how we could split mdl.xml into smaller, more manageable chunks. The patch (forthcoming) simply iterates over the files in $CHUKWA_CONF_DIR and if the file name ends with "mdl.xml", it will be added as a resource to the Configuration object used (mdl.xml is loaded first and the others are loaded subsequently, ensuring that mdl.xml is not then loaded again).

      One likely concern is that this doesn't follow the default/site method of specifying overrides. Also, the ordering of the configuration files' inclusionis based on the JVM and/or platform implementation of java.io.File.listFiles.

      Regardless, this made it very simple to isolate the files needed by each data type and update them as needed.

        Activity

        Hide
        kirktrue Kirk True added a comment -

        Changes to DataConfig and DatabaseConfig to allow overrides to the mdl.xml to include other configuration files.

        Show
        kirktrue Kirk True added a comment - Changes to DataConfig and DatabaseConfig to allow overrides to the mdl.xml to include other configuration files.
        Hide
        eyang Eric Yang added a comment -

        +1 Looks good.

        Show
        eyang Eric Yang added a comment - +1 Looks good.
        Hide
        eyang Eric Yang added a comment -

        I just committed this, thanks Kirk.

        Show
        eyang Eric Yang added a comment - I just committed this, thanks Kirk.
        Hide
        hudson Hudson added a comment -

        Integrated in Chukwa-trunk #401 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/401/)
        Updated Change log for CHUKWA-489 and CHUKWA-490.
        CHUKWA-490. Enabling mdl.xml to be split into multiple files for ease of maintenance. (Kirk True via Eric Yang)

        Show
        hudson Hudson added a comment - Integrated in Chukwa-trunk #401 (See http://hudson.zones.apache.org/hudson/job/Chukwa-trunk/401/ ) Updated Change log for CHUKWA-489 and CHUKWA-490 . CHUKWA-490 . Enabling mdl.xml to be split into multiple files for ease of maintenance. (Kirk True via Eric Yang)

          People

          • Assignee:
            Unassigned
            Reporter:
            kirktrue Kirk True
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development