Uploaded image for project: 'REEF'
  1. REEF
  2. REEF-751

Add a standard way to provide cluster-local configuration

    Details

      Description

      When running a REEF application on a specific cluster, we sometimes have to make cluster-specific configuration choices beyond what's available via e.g. the YARN Configuration. One example are clusters which only allow us to open TCP ports in a specific range.

      Right now, we require the application to add such configuration to the Driver, e.g. via the DriverConfigurationProviders mechanism. This is awkward, because it limits the portability of applications: In order for an app to run such a cluster, one would have to change the app's code. This is undesirable.

      Instead, we should have a standard mechanism by which one can provide additional runtime-level configuration to be picked up by the REEF client. Let's use this JIRA to discuss potential solutions to this issue.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              markus.weimer Markus Weimer
            • Votes:
              1 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: