Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-2973

Use LocalRelation for all ExecutedCommands, avoid job for take/collect()

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Critical
    • Resolution: Later
    • None
    • None
    • SQL
    • None

    Description

      Right now, sql("show tables").collect() will start a Spark job which shows up in the UI. There should be a way to get these without this step.

      Attachments

        Issue Links

          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. SPARK-8094 Calling take on ExecutedCommand and OneRowRelation should not trigger job execution

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          links to

          Pull request #2215 [Github] Pull Request #2215 (liancheng)

          Pull request #5247 [Github] Pull Request #5247 (scwf)

          Activity

            People

              lian cheng Cheng Lian
              ilikerps Aaron Davidson
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: