Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23089

"Unable to create operation log session directory" when parent directory not present

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.3.0
    • Component/s: SQL
    • Labels:
      None
    • Environment:

      Description

      When creating a session directory, Thrift should create the parent directory (i.e. /tmp/hive/operation_logs) if it is not present.

      It's common for operators to clean-up old and empty directories in /tmp, or to have tools (systemd-tmpfiles or tmpwatch) that do it automatically.

      This was fixed in HIVE-12262 but not in Spark Thrift as seen by this:

      18/01/15 14:22:49 WARN HiveSessionImpl: Unable to create operation log session directory: /tmp/hive/operation_logs/683a6318-adc4-42c4-b665-11dad14d7ec7

      Resolved by manually creating /tmp/hive/operation_logs/

        Attachments

          Activity

            People

            • Assignee:
              mgaido Marco Gaido
              Reporter:
              seano Sean Roberts
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: