Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2442

Need an alternative to providing clear text passwords in flume config

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.5.0.1
    • 1.9.0
    • Sinks+Sources

    Description

      For some sources and sinks, currently, passwords to keystores/other are specified in clear text in the flume config file. Since flume config files are often easily accessible to a broader audience (like in source control for instance), the visibility of these passwords can be too much and risky for institutions where security is too critical (like banks)

      To help address this visibility issue it would be desirable to do the following two things :

      1) Store the password in a separate file and provide the path of that password file in the flume config. this will enable the flume config to be shared with a wider audience and reduce risk. the password file will need to be very tightly guarded. Some components like file channel & JMS source already do this.

      2) As an additional measure, obfuscate the password in the external password file. A simple command line tool can be used to generate the obfuscated password file. Flume source/sink configuration will read the password file and de-obfuscate the password before using it to access the keystore. This obfuscation step IMO is nice but unclear to me if it is essential.

      The following Sources and Sinks appear to use inline cleartext passwords:

      • Avro Source
      • Avro sink
      • HTTP(S) source
      • File Channel
      • JMS Source

      JDBC channel also uses inline passwords but i am not aware of anybody who uses JDBC channel. So it may not be an issue.

      Attachments

        1. FLUME-2442.patch.7
          62 kB
          Balázs Donát Bessenyei
        2. FLUME-2442.patch.9
          63 kB
          Balázs Donát Bessenyei
        3. FLUME-2442.v1.patch
          62 kB
          Roshan Naik
        4. FLUME-2442.v2.patch
          62 kB
          Roshan Naik
        5. FLUME-2442.v3.patch
          62 kB
          Roshan Naik
        6. FLUME-2442.v4.patch
          63 kB
          Roshan Naik
        7. FLUME-2442.v5.patch
          63 kB
          Roshan Naik

        Activity

          People

            venkatnrangan Venkat Ranganathan
            roshan_naik Roshan Naik
            Votes:
            1 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: