Uploaded image for project: 'Apache Apex Core'
  1. Apache Apex Core
  2. APEXCORE-234

Better application configuration specification

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      The current property file specification follows the hadoop configuration file format and this has led to some drawbacks.
      a. The names for the properties and attributes are verbose in the configuration file.
      b. When there are nested properties in operators the syntax deviates from the bean specification because it introduces some specific keywords in the specification like .prop and ,attr.

      There will already be some changes afoot based on the following
      a. When adding ability to specify single instance attributes (https://malhar.atlassian.net/browse/APEXCORE-233) implementing it in the current syntax may not be possible or lead to very unwieldy syntax.
      b. There are also other ideas such as one from David to have the ability to specify global application level attributes which possible require rethinking the current syntax.

      Users have also asked for an easier and more consistent way to specify these properties. This issue is to track the ideas and progress of these changes.

        Issue Links

          Activity

          Hide
          thw Thomas Weise added a comment -

          I think that the current approach of setting properties on operators, while flexible, does not provide the required level of abstraction. There should be support for application level properties, including complex objects, and the application assembly should set these properties on whatever operators need them. This is pretty much what happens with modules today, and perhaps applications can be seen like modules without ports?

          JSON or YAML might be good formats to configure applications.

          Show
          thw Thomas Weise added a comment - I think that the current approach of setting properties on operators, while flexible, does not provide the required level of abstraction. There should be support for application level properties, including complex objects, and the application assembly should set these properties on whatever operators need them. This is pretty much what happens with modules today, and perhaps applications can be seen like modules without ports? JSON or YAML might be good formats to configure applications.
          Hide
          gaurav.gopi123 gaurav gupta added a comment -

          Pramod,

          Apex should also support properties inheritance. For e.g. an app can have a some common properties and some environment specific properties. Common properties should be defined in one file and environment specific properties should just refer them instead of defining them again

          Show
          gaurav.gopi123 gaurav gupta added a comment - Pramod, Apex should also support properties inheritance. For e.g. an app can have a some common properties and some environment specific properties. Common properties should be defined in one file and environment specific properties should just refer them instead of defining them again
          Hide
          dtram Munagala V. Ramanath added a comment -

          This issue should be addressed.

          I've long felt that simple declarative configuration files ought to be a thing of the past. We would
          have a lot more flexibility if all configuration files were scripts executable by a JVM. Then, we could
          add arbitrary logic there to provide substantially more flexible configuration. We just need to pick a
          language that has a JVM implementation (e.g. JavaScript, JRuby, Groovy, etc.).

          There will be some memory overhead for the language execution engine of course but the flexibility
          makes it worthwhile.

          Show
          dtram Munagala V. Ramanath added a comment - This issue should be addressed. I've long felt that simple declarative configuration files ought to be a thing of the past. We would have a lot more flexibility if all configuration files were scripts executable by a JVM. Then, we could add arbitrary logic there to provide substantially more flexible configuration. We just need to pick a language that has a JVM implementation (e.g. JavaScript, JRuby, Groovy, etc.). There will be some memory overhead for the language execution engine of course but the flexibility makes it worthwhile.

            People

            • Assignee:
              Unassigned
              Reporter:
              PramodSSImmaneni Pramod Immaneni
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:

                Development