Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-12609

Make R to JVM timeout configurable

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • None
    • None
    • SparkR

    Description

      The timeout from R to the JVM is hardcoded at 6000 seconds in https://github.com/apache/spark/blob/6c5bbd628aaedb6efb44c15f816fea8fb600decc/R/pkg/R/client.R#L22

      This results in Spark jobs that take more than 100 minutes to always fail. We should make this timeout configurable through SparkConf.

      Attachments

        Issue Links

          duplicates

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. SPARK-17919 Make timeout to RBackend configurable in SparkR

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              shivaram Shivaram Venkataraman
              Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: