Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-293

Redirect Spark Log to REST Response Log field instead of redirecting to livi-server.out log file

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.3
    • Fix Version/s: 0.4.0
    • Component/s: Core, REPL, RSC, Server
    • Labels:
      None

      Description

      Currently, Livy redirects the spark log to $LIVY_HOME//loglivy-livy-server.out log file for Interactive Sessions.

      So, when a session fails before initiating the SparkContext, then user doesn't have a way to know why his job got failed unless he is going to $LIVY_HOME//loglivy-livy-server.out to debug. But, all users are not going to have access to that log file.

      Hence, it would be better if we redirect the spark log lines to REST response log field for debugging (as how we are doing for Batch session).

        Attachments

          Activity

            People

            • Assignee:
              prabhu1984 Prabhu Kasinathan
              Reporter:
              prabhu1984 Prabhu Kasinathan
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: