Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2993

Support environment variables in configuration files

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.8.0
    • Component/s: Configuration
    • Labels:
      None

      Description

      Flume does not currently support environment variable interpolation in the properties file configuration.

      Enabling it would help with

      • removing security credentials from config files
      • help with copy-pastes in configuration files with multiple agents defined

      It is arguably a best practice to store (some) config in the environment variables: https://12factor.net/config

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          ABORTED: Integrated in Jenkins build Flume-trunk-hbase-1 #236 (See https://builds.apache.org/job/Flume-trunk-hbase-1/236/)
          FLUME-2993. Add support for environment variables in configuration files (bessbd: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=5e52ac4ad18a7776f4c74499c1314fbfd3513160)

          • (edit) flume-ng-node/src/main/java/org/apache/flume/node/PropertiesFileConfigurationProvider.java
          • (add) flume-ng-node/src/main/java/org/apache/flume/node/EnvVarResolverProperties.java
          • (edit) flume-ng-doc/sphinx/FlumeUserGuide.rst
          • (edit) flume-ng-node/pom.xml
          • (add) flume-ng-node/src/test/java/org/apache/flume/node/TestEnvVarResolverProperties.java
          • (add) flume-ng-node/src/test/resources/flume-conf-with-envvars.properties
          Show
          hudson Hudson added a comment - ABORTED: Integrated in Jenkins build Flume-trunk-hbase-1 #236 (See https://builds.apache.org/job/Flume-trunk-hbase-1/236/ ) FLUME-2993 . Add support for environment variables in configuration files (bessbd: http://git-wip-us.apache.org/repos/asf/flume/repo?p=flume.git&a=commit&h=5e52ac4ad18a7776f4c74499c1314fbfd3513160 ) (edit) flume-ng-node/src/main/java/org/apache/flume/node/PropertiesFileConfigurationProvider.java (add) flume-ng-node/src/main/java/org/apache/flume/node/EnvVarResolverProperties.java (edit) flume-ng-doc/sphinx/FlumeUserGuide.rst (edit) flume-ng-node/pom.xml (add) flume-ng-node/src/test/java/org/apache/flume/node/TestEnvVarResolverProperties.java (add) flume-ng-node/src/test/resources/flume-conf-with-envvars.properties
          Hide
          bessbd Bessenyei Balázs Donát added a comment -

          Lior Zeno: thank you for the review!

          Show
          bessbd Bessenyei Balázs Donát added a comment - Lior Zeno : thank you for the review!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/flume/pull/97

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/flume/pull/97
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 5e52ac4ad18a7776f4c74499c1314fbfd3513160 in flume's branch refs/heads/trunk from Bessenyei Balázs Donát
          [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=5e52ac4 ]

          FLUME-2993. Add support for environment variables in configuration files

          Flume does not currently support environment variable interpolation in the properties file configuration.

          Enabling it helps with:

          • removing security credentials from config files
          • copy-pastes in configuration files when defining multiple agents

          This closes #97

          Reviewers: Lior Zeno, Jeff Holoman, Shang Wu

          (Bessenyei Balázs Donát via Bessenyei Balázs Donát)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 5e52ac4ad18a7776f4c74499c1314fbfd3513160 in flume's branch refs/heads/trunk from Bessenyei Balázs Donát [ https://git-wip-us.apache.org/repos/asf?p=flume.git;h=5e52ac4 ] FLUME-2993 . Add support for environment variables in configuration files Flume does not currently support environment variable interpolation in the properties file configuration. Enabling it helps with: removing security credentials from config files copy-pastes in configuration files when defining multiple agents This closes #97 Reviewers: Lior Zeno, Jeff Holoman, Shang Wu (Bessenyei Balázs Donát via Bessenyei Balázs Donát)
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user bessbd opened a pull request:

          https://github.com/apache/flume/pull/97

          FLUME-2993. Add support for environment variables in configuration files

          Flume does not currently support environment variable interpolation in the properties file configuration.

          Enabling it helps with:

          • removing security credentials from config files
          • copy-pastes in configuration files when defining multiple agents

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/bessbd/flume FLUME-2993

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/flume/pull/97.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #97


          commit 0f0dea972ad56b9799d858218f1fff371dd698e5
          Author: Bessenyei Balázs Donát <bessbd@gmail.com>
          Date: 2016-12-03T21:21:48Z

          FLUME-2993. Add support for environment variables in configuration files

          Flume does not currently support environment variable interpolation in the properties file configuration.

          Enabling it helps with:

          • removing security credentials from config files
          • copy-pastes in configuration files when defining multiple agents

          Show
          githubbot ASF GitHub Bot added a comment - GitHub user bessbd opened a pull request: https://github.com/apache/flume/pull/97 FLUME-2993 . Add support for environment variables in configuration files Flume does not currently support environment variable interpolation in the properties file configuration. Enabling it helps with: removing security credentials from config files copy-pastes in configuration files when defining multiple agents You can merge this pull request into a Git repository by running: $ git pull https://github.com/bessbd/flume FLUME-2993 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flume/pull/97.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #97 commit 0f0dea972ad56b9799d858218f1fff371dd698e5 Author: Bessenyei Balázs Donát <bessbd@gmail.com> Date: 2016-12-03T21:21:48Z FLUME-2993 . Add support for environment variables in configuration files Flume does not currently support environment variable interpolation in the properties file configuration. Enabling it helps with: removing security credentials from config files copy-pastes in configuration files when defining multiple agents
          Hide
          bessbd Bessenyei Balázs Donát added a comment -

          This has been around for a while.
          If someone has review comments, please share. Otherwise, I'll commit this soon.

          Show
          bessbd Bessenyei Balázs Donát added a comment - This has been around for a while. If someone has review comments, please share. Otherwise, I'll commit this soon.
          Hide
          bessbd Bessenyei Balázs Donát added a comment -

          I have created and linked a reviewboard entry.

          Show
          bessbd Bessenyei Balázs Donát added a comment - I have created and linked a reviewboard entry.

            People

            • Assignee:
              bessbd Bessenyei Balázs Donát
              Reporter:
              bessbd Bessenyei Balázs Donát
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development