Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-33605

Add gcs-connector to hadoop-cloud module

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.4.0
    • 3.4.0
    • Build
    • None

    Description

      Spark comes with some S3 batteries included, which makes it easier to use with S3, for GCS to work users are required to manually configure the jars. This is especially problematic for python users who may not be accustomed to java dependencies etc. This is an example of workaround for pyspark: pyspark_gcs. If we include the GCS connector, it would make things easier for GCS users.

      Please let me know what you think.

      Attachments

        Activity

          People

            dongjoon Dongjoon Hyun
            ravwojdyla Rafal Wojdyla
            Votes:
            2 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: