Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2803

Mask passwords when printing out configs/args in MapReduceMain and SparkMain

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • 5.0.0b1, 4.3.1
    • action
    • None

    Description

      Sometimes passwords are displayed in both MapReduce and Spark action.

      MapReduce: when using HADOOP_CREDSTORE_PASSWORD, it must be passed to some Hadoop-specific config values, like mapred.child.env. This is easy to fix because we already have a method logMasking() where you can define a maskSet which contains a list of property keys to be masked.

      Note that this is not necessarily the perfect solution, since you can pass multiple env. vars separated by a colon, and only the password specific parts should be masked. But we need a working solution relatively quickly - later we can enhance this, eg. we can re-use PasswordMasker in some way (right now it only works with Map<String, String>).

      Spark: for Spark, we have to pass passwords like this:

      --conf spark.executorEnv.HADOOP_CREDSTORE_PASSWORD=<custom keystore password>

      The Spark arguments are printed in SparkMain.run(). There is already a code in LauncherMapper.printArgs() which deals with situations like this, but it's not perfect because it only works if the args look something like --password pwd123. So if a single arg contains a password, it doesn't work, therefore we need a different approach here.

      Attachments

        1. OOZIE-2803-001.patch
          45 kB
          Peter Bacsko
        2. OOZIE-2803-002.patch
          45 kB
          Peter Bacsko
        3. OOZIE-2803-003.patch
          45 kB
          Peter Bacsko
        4. OOZIE-2803-004.patch
          45 kB
          Peter Bacsko
        5. OOZIE-2803-005.patch
          45 kB
          Peter Bacsko

        Activity

          People

            pbacsko Peter Bacsko
            pbacsko Peter Bacsko
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: