Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-2803

Mask passwords when printing out configs/args in MapReduceMain and SparkMain

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0.0b1, 4.3.1
    • Component/s: action
    • Labels:
      None

      Description

      Sometimes passwords are displayed in both MapReduce and Spark action.

      MapReduce: when using HADOOP_CREDSTORE_PASSWORD, it must be passed to some Hadoop-specific config values, like mapred.child.env. This is easy to fix because we already have a method logMasking() where you can define a maskSet which contains a list of property keys to be masked.

      Note that this is not necessarily the perfect solution, since you can pass multiple env. vars separated by a colon, and only the password specific parts should be masked. But we need a working solution relatively quickly - later we can enhance this, eg. we can re-use PasswordMasker in some way (right now it only works with Map<String, String>).

      Spark: for Spark, we have to pass passwords like this:

      --conf spark.executorEnv.HADOOP_CREDSTORE_PASSWORD=<custom keystore password>

      The Spark arguments are printed in SparkMain.run(). There is already a code in LauncherMapper.printArgs() which deals with situations like this, but it's not perfect because it only works if the args look something like --password pwd123. So if a single arg contains a password, it doesn't work, therefore we need a different approach here.

        Attachments

        1. OOZIE-2803-005.patch
          45 kB
          Peter Bacsko
        2. OOZIE-2803-004.patch
          45 kB
          Peter Bacsko
        3. OOZIE-2803-003.patch
          45 kB
          Peter Bacsko
        4. OOZIE-2803-002.patch
          45 kB
          Peter Bacsko
        5. OOZIE-2803-001.patch
          45 kB
          Peter Bacsko

          Activity

            People

            • Assignee:
              pbacsko Peter Bacsko
              Reporter:
              pbacsko Peter Bacsko
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: