Uploaded image for project: 'Oozie'
  1. Oozie
  2. OOZIE-1814

Oozie should mask any passwords in logs and REST interfaces

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: trunk
    • Fix Version/s: 4.3.0
    • Component/s: None
    • Labels:

      Description

      Following passwords are currently visible in the instrumentation log, REST endpoints, WebUI, and CLI (WebUI and CLI simply call the REST endpoints):

      • javax.net.ssl.trustStorePassword
      • oozie.https.keystore.pass
      • HADOOP_CREDSTORE_PASSWORD
      • OOZIE_HTTPS_KEYSTORE_PASSWORD
      • OOZIE_HTTPS_TRUSTSTORE_PASSWORD

      There are a few examples that illustrate password leakage.

      # grep -i pass /var/log/oozie/oozie-instrumentation.log
          OOZIE_HTTPS_TRUSTSTORE_PASSWORD = password
          javax.net.ssl.trustStorePassword = password
          oozie.https.keystore.pass = password
          HADOOP_CREDSTORE_PASSWORD = password
          OOZIE_HTTPS_KEYSTORE_PASSWORD = password
          CATALINA_OPTS = -Xms603979776 -Xmx603979776 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/OOZIE-1_OOZIE-1-OOZIE_SERVER-2e75cc1293d9058eef7250a18f347c43_pid30867.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Doozie.home.dir=/usr/lib/oozie -Doozie.config.dir=/var/run/cloudera-scm-agent/process/320-oozie-OOZIE_SERVER -Doozie.log.dir=/var/log/oozie -Doozie.log.file=oozie-cmf-OOZIE-1-OOZIE_SERVER-nightly-1.gce.cloudera.com.log.out -Doozie.config.file=oozie-site.xml -Doozie.log4j.file=log4j.properties -Doozie.log4j.reload=10 -Doozie.http.hostname=nightly-1.gce.cloudera.com -Doozie.http.port=11000 -Djava.net.preferIPv4Stack=true -Doozie.admin.port=11001 -Dderby.stream.error.file=/var/log/oozie/derby.log -Doozie.instance.id=nightly-1.gce.cloudera.com -Djava.library.path=/usr/lib/hadoop/lib/native -Doozie.https.port=11443 -Djavax.net.ssl.trustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks -Djavax.net.ssl.trustStorePassword=password
      

      Oozie dumps the env vars and Java sys props to the instrumentation log on startup.

      # curl --negotiate -u foo:bar -k https://nightly-1.gce.cloudera.com:11443/oozie/v2/admin/os-env | python -m json.tool | grep -i pass
      "CATALINA_OPTS": "-Xms603979776 -Xmx603979776 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/OOZIE-1_OOZIE-1-OOZIE_SERVER-2e75cc1293d9058eef7250a18f347c43_pid30867.hprof -XX:OnOutOfMemoryError=/usr/lib64/cmf/service/common/killparent.sh -Doozie.home.dir=/usr/lib/oozie -Doozie.config.dir=/var/run/cloudera-scm-agent/process/320-oozie-OOZIE_SERVER -Doozie.log.dir=/var/log/oozie -Doozie.log.file=oozie-cmf-OOZIE-1-OOZIE_SERVER-nightly-1.gce.cloudera.com.log.out -Doozie.config.file=oozie-site.xml -Doozie.log4j.file=log4j.properties -Doozie.log4j.reload=10 -Doozie.http.hostname=nightly-1.gce.cloudera.com -Doozie.http.port=11000 -Djava.net.preferIPv4Stack=true -Doozie.admin.port=11001 -Dderby.stream.error.file=/var/log/oozie/derby.log -Doozie.instance.id=nightly-1.gce.cloudera.com -Djava.library.path=/usr/lib/hadoop/lib/native -Doozie.https.port=11443 -Djavax.net.ssl.trustStore=/etc/cdep-ssl-conf/CA_STANDARD/truststore.jks -Djavax.net.ssl.trustStorePassword=password ",
          "HADOOP_CREDSTORE_PASSWORD": "password",
          "OOZIE_HTTPS_KEYSTORE_PASSWORD": "password",
          "OOZIE_HTTPS_TRUSTSTORE_PASSWORD": "password",
      
      # curl --negotiate -u foo:bar -k https://nightly-1.gce.cloudera.com:11443/oozie/v2/admin/java-sys-properties | python -m json.tool | grep -i pass
          "javax.net.ssl.trustStorePassword": "password",
          "oozie.https.keystore.pass": "password",
      

      The REST API has two endpoints, admin/os-env and admin/java-sys-properties, which are also available in the Web UI and CLI. These expose the env vars and Java sys props too.

      We should mask these like we do for the configuration endpoint.

        Attachments

        1. oozie-1814.patch
          1 kB
          Bowen Zhang
        2. OOZIE-1814.002.patch
          19 kB
          Andras Piros
        3. OOZIE-1814.003.patch
          21 kB
          Andras Piros
        4. OOZIE-1814.004.patch
          21 kB
          Andras Piros
        5. OOZIE-1814.005.patch
          22 kB
          Andras Piros
        6. OOZIE-1814.006.patch
          23 kB
          Andras Piros
        7. OOZIE-1814.007.patch
          23 kB
          Andras Piros

          Activity

            People

            • Assignee:
              andras.piros Andras Piros
              Reporter:
              bowenzhangusa Bowen Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: