Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-11929

spark-shell log level customization is lost if user provides a log4j.properties file

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: Spark Core, Spark Shell
    • Labels:
      None

      Description

      Logging.scala has code that defines the default log level for the spark-shell to WARN, to avoid lots of noise in the output.

      But if a user provides a log4j.properies file in the Spark configuration, that customization is lost. That means that without a log4j.properties, there are two different configurations (one for regular apps, one for the shell). But if you have a custom file, you lose the ability to easily differentiate between those two, and you're stuck with a single config for both.

      It would be nice to allow different configurations also in the second case.

        Attachments

          Activity

            People

            • Assignee:
              vanzin Marcelo Masiero Vanzin
              Reporter:
              vanzin Marcelo Masiero Vanzin
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: