Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22406

pyspark version tag is wrong on PyPi

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.2.0
    • Fix Version/s: 2.1.2, 2.2.1
    • Component/s: PySpark
    • Labels:
      None
    • Target Version/s:

      Description

      On pypi.python.org, the pyspark package is tagged with version 2.2.0.post0: https://pypi.python.org/pypi/pyspark/2.2.0

      However, when you install the package, it has version 2.2.0.

      This has really annoying consequences: if you try pip install pyspark==2.2.0, it won't work. Instead you have to do pip install pyspark==2.2.0.post0. Then, if you later run the same command (pip install pyspark==2.2.0.post0), it won't recognize the existing pyspark installation (because it has version 2.2.0) and instead will reinstall it, which is very slow because pyspark is a large package.

      This can happen if you add a new package to a requirements.txt file; you end up waiting a lot longer than necessary because every time you run pip install -r requirements.txt it reinstalls pyspark.

      Can you please change the package on PyPi to have the version 2.2.0?

        Attachments

          Activity

            People

            • Assignee:
              holden Holden Karau
              Reporter:
              kerrick-lyft Kerrick Staley
              Shepherd:
              Holden Karau
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: