Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22302

Remove manual backports for subprocess.check_output and check_call

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Trivial
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.3.0
    • Component/s: Project Infra
    • Labels:
      None

      Description

      This JIRA is loosely related with SPARK-21573. Python 2.6 could be used in Jenkins given the past cases and investigations up to my knowledge and it looks failing to execute some other scripts.

      In this particular case, it was:

      cd dev && python2.6
      
      >>> from sparktestsupport import shellutils
      >>> shellutils.subprocess_check_call("ls")
      Traceback (most recent call last):
        File "<stdin>", line 1, in <module>
        File "sparktestsupport/shellutils.py", line 46, in subprocess_check_call
          retcode = call(*popenargs, **kwargs)
      NameError: global name 'call' is not defined
      

      Please see https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3950/console

      Since we dropped the Python 2.6.x support, looks better we remove those workarounds and print out explicit error messages in order to duplicate the efforts to find out the root causes for such cases.

        Attachments

          Activity

            People

            • Assignee:
              hyukjin.kwon Hyukjin Kwon
              Reporter:
              hyukjin.kwon Hyukjin Kwon
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: