Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.8.0
    • Component/s: core
    • Labels:
      None

      Description

      Service configuration is currently done via the Whirr properties file by using a special per-service prefix (e.g. 'hadoop-common'). Using a templating approach (using e.g. Velocity) would make changing properties more familiar to users, and would make it easier for service implementors (since they wouldn't need to implement a generator class for each configuration file).

      It might even be possible to support the old syntax (so users can still configure everything from one file if they wished) by, e.g., appending all properties starting 'hadoop-common' to the core-site.xml template, etc.

      1. WHIRR-370.patch
        48 kB
        Karel Vervaeke
      2. WHIRR-370.patch
        48 kB
        Andrei Savu
      3. WHIRR-370.patch
        47 kB
        Karel Vervaeke
      4. WHIRR-370.patch
        47 kB
        Karel Vervaeke
      5. WHIRR-370.patch
        37 kB
        Andrei Savu
      6. WHIRR-370.patch
        36 kB
        Karel Vervaeke

        Issue Links

          Activity

          Hide
          Karel Vervaeke added a comment -

          Committed to trunk

          Show
          Karel Vervaeke added a comment - Committed to trunk
          Hide
          Karel Vervaeke added a comment -

          Reverted the setTCP(false) change

          Show
          Karel Vervaeke added a comment - Reverted the setTCP(false) change
          Hide
          Andrei Savu added a comment -

          How about creating an option for TCP/UDP DNS switch? (in another JIRA). Or we need to refactor that code to be able to use the hostname information provided by Amazon through the API.

          Show
          Andrei Savu added a comment - How about creating an option for TCP/UDP DNS switch? (in another JIRA). Or we need to refactor that code to be able to use the hostname information provided by Amazon through the API.
          Hide
          Karel Vervaeke added a comment -

          Sorry about the TCP/DNS switch. I have to switch it to UDP for testing at home because my router doesn't support TCP.
          Should totally not be in the patch.

          Show
          Karel Vervaeke added a comment - Sorry about the TCP/DNS switch. I have to switch it to UDP for testing at home because my router doesn't support TCP. Should totally not be in the patch.
          Hide
          Andrei Savu added a comment -

          Updated to work with trunk. Karel let me know what you think.

          I had some issues while running the Hadoop itests on Amazon - I suspect the change from TCP to UDP for DNS. I will try again later today.

          Show
          Andrei Savu added a comment - Updated to work with trunk. Karel let me know what you think. I had some issues while running the Hadoop itests on Amazon - I suspect the change from TCP to UDP for DNS. I will try again later today.
          Hide
          Andrei Savu added a comment -

          +1 (the conflict with WHIRR-504 is easy to fix - only imports).

          Show
          Andrei Savu added a comment - +1 (the conflict with WHIRR-504 is easy to fix - only imports).
          Hide
          Karel Vervaeke added a comment -

          Once more to make checkstyle happy

          Show
          Karel Vervaeke added a comment - Once more to make checkstyle happy
          Hide
          Karel Vervaeke added a comment -

          Updated patch gets rid of velocity.log files

          Show
          Karel Vervaeke added a comment - Updated patch gets rid of velocity.log files
          Hide
          Andrei Savu added a comment -

          Can we change the logging settings for Velocity? I see two news files created after running the unit tests: services/chef/velocity.log and services/hadoop/velocity.log. Overall I'm +1 for committing this change.

          Show
          Andrei Savu added a comment - Can we change the logging settings for Velocity? I see two news files created after running the unit tests: services/chef/velocity.log and services/hadoop/velocity.log. Overall I'm +1 for committing this change.
          Hide
          Andrei Savu added a comment -

          Updated existing patch to work with Karaf. I have edited platforms/karaf/feature/src/main/resources/features.xml. All hadoop integration tests work for me on aws-ec2.

          Show
          Andrei Savu added a comment - Updated existing patch to work with Karaf. I have edited platforms/karaf/feature/src/main/resources/features.xml. All hadoop integration tests work for me on aws-ec2.
          Hide
          Karel Vervaeke added a comment -

          Patch introduces velocity, exposed via ClusterActionEvent.getTemplateEngine().

          The patch also fixes WHIRR-561 (ganglia configuration).

          The user can use custom templates by putting the template in the classpath (e.g. $WHIRR_HOME/conf) and adding a pointer in the recipe (e.g. hadoop-metrics.template=my-custom-hadoop-metrics.properties.vm

          Notes / possible improvements:

          • whirr-hadoop now has a test dependency on whirr-ganglia (which is kinda ugly)
          • support adding templates via non-classpath locations (generally, support changing the velocity engine configuration, which should be easy since it's configured via a property object).
          • support multiple template engines via plugin mechanism? (meh)
          Show
          Karel Vervaeke added a comment - Patch introduces velocity, exposed via ClusterActionEvent.getTemplateEngine(). The patch also fixes WHIRR-561 (ganglia configuration). The user can use custom templates by putting the template in the classpath (e.g. $WHIRR_HOME/conf) and adding a pointer in the recipe (e.g. hadoop-metrics.template=my-custom-hadoop-metrics.properties.vm Notes / possible improvements: whirr-hadoop now has a test dependency on whirr-ganglia (which is kinda ugly) support adding templates via non-classpath locations (generally, support changing the velocity engine configuration, which should be easy since it's configured via a property object). support multiple template engines via plugin mechanism? (meh)
          Hide
          Tom White added a comment -

          Velocity is tried and tested, so would be a good choice in my opinion.

          Show
          Tom White added a comment - Velocity is tried and tested, so would be a good choice in my opinion.
          Hide
          Andrei Savu added a comment -

          Let's decide on what we use. Is Velocity our only choice here? Any other similar project at Apache?

          Show
          Andrei Savu added a comment - Let's decide on what we use. Is Velocity our only choice here? Any other similar project at Apache?

            People

            • Assignee:
              Unassigned
              Reporter:
              Tom White
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development