Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-41286

Build, package and infrastructure for Spark Connect

    XMLWordPrintableJSON

Details

    • Umbrella
    • Status: Resolved
    • Critical
    • Resolution: Done
    • 3.4.0
    • None
    • Connect
    • None

    Attachments

      1.
      Check the generated python protos in GitHub Actions Sub-task Resolved Ruifeng Zheng
      2.
      Enforce Scalafmt for Spark Connect Module Sub-task Resolved Martin Grund
      3.
      pin 'grpcio==1.48.1' 'protobuf==4.21.6' Sub-task Resolved Ruifeng Zheng
      4.
      Homogenize the protobuf version across server and client Sub-task Resolved Martin Grund
      5.
      protoc-3.21.9-linux-x86_64.exe requires GLIBC_2.14 Sub-task Resolved Haonan Jiang
      6.
      Upgrade buf to v1.9.0 Sub-task Resolved Ruifeng Zheng
      7.
      Check and upgrade buf.build/protocolbuffers/plugins/python to 3.19.5 Sub-task Resolved Bjørn Jørgensen
      8.
      Add `CONNECT` label Sub-task Resolved Hyukjin Kwon
      9.
      protoc-3.21.1-linux-x86_64.exe requires GLIBC_2.14 Sub-task Resolved Yang Jie
      10.
      Update sbt-protoc to 1.0.6 Sub-task Resolved Yuming Wang
      11.
      Avoid embedding Spark Connect in the Apache Spark binary release Sub-task Resolved Hyukjin Kwon
      12.
      Shade more dependency to be able to run separately Sub-task Resolved Hyukjin Kwon
      13.
      Avoid hardcoded versions in SBT build Sub-task Resolved Hyukjin Kwon
      14.
      Replace shaded netty with grpc netty to avoid double shaded dependency. Sub-task Resolved Martin Grund
      15.
      Re-generate Spark Connect Python protos Sub-task Resolved Martin Grund
      16.
      Sharing namespace between PySpark and Spark Connect Sub-task Resolved Hyukjin Kwon
      17.
      Add Spark Connect and protobuf into setup.py with specifying dependencies Sub-task Resolved Hyukjin Kwon
      18.
      Upgrade buf to v1.11.0 Sub-task Resolved Ruifeng Zheng
      19.
      Support PyPI packaging without JVM Sub-task Resolved Unassigned
      20.
      Distribution build for Spark Connect does not work with Spark Shell Sub-task Resolved Martin Grund
      21.
      Upgrade buf to v1.15.0 Sub-task Resolved Ruifeng Zheng
      22.
      Upgrade buf to v1.15.1 Sub-task Resolved Pan Bingkun

      Activity

        People

          gurwls223 Hyukjin Kwon
          gurwls223 Hyukjin Kwon
          Hyukjin Kwon Hyukjin Kwon
          Votes:
          0 Vote for this issue
          Watchers:
          1 Start watching this issue

          Dates

            Created:
            Updated:
            Resolved: