Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-8821

The ec2 script doesn't run on python 3 with an utf8 env

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.0
    • Fix Version/s: 1.4.1, 1.5.0
    • Component/s: EC2
    • Labels:
      None
    • Environment:

      Archlinux, UTF8 LANG env

    • Flags:
      Patch

      Description

      Otherwise the script will crash with

      • Downloading boto...
        Traceback (most recent call last):
        File "ec2/spark_ec2.py", line 148, in <module>
        setup_external_libs(external_libs)
        File "ec2/spark_ec2.py", line 128, in setup_external_libs
        if hashlib.md5(tar.read()).hexdigest() != lib["md5"]:
        File "/usr/lib/python3.4/codecs.py", line 319, in decode
        (result, consumed) = self._buffer_decode(data, self.errors, final)
        UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte

      In case of an utf8 env setting.

        Attachments

          Activity

            People

            • Assignee:
              reactormonk Simon Hafner
              Reporter:
              reactormonk Simon Hafner
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: