Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6952

spark-daemon.sh PID reuse check fails on long classpath

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 1.2.2, 1.3.1
    • Fix Version/s: 1.2.3, 1.3.2, 1.4.0
    • Component/s: Deploy
    • Labels:
      None

      Description

      sbin/spark-daemon.sh uses ps -p "$TARGET_PID" -o args= to figure out whether the process running with the expected PID is actually a Spark daemon. When running with a large classpath, the output of ps gets truncated and the check fails spuriously.

      I think we should weaken the check to see if it's a java command (which is something we do in other parts of the script) rather than looking for the specific main class name. This means that SPARK-4832 might happen under a slightly broader range of circumstances (a java program happened to reuse the same PID), but it seems worthwhile compared to failing consistently with a large classpath.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                punya Punya Biswal
                Reporter:
                punya Punya Biswal
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: