Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-7712

Impala read from and write to GCS

    XMLWordPrintableJSON

    Details

    • Epic Color:
      ghx-label-4

      Description

      Can Impala read from and write to google cloud storage GCS like the way it does with amazon s3

      I have tested the use case with S3, but when talking to GCS impala errors out with:
      Query: create table gcs_impala2 (title string) location 'gs://mybucket-gcs/some_data/' ERROR: AnalysisException: null CAUSED BY: RuntimeException: java.lang.ClassNotFoundException: Class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem not found CAUSED BY: ClassNotFoundException: Class com.google.cloud.hadoop.fs.gcs.GoogleHadoopFileSystem not found
       On the same cluster i have Hive talking to GCS using the GCS connector jar provided by google form :
      https://cloud.google.com/dataproc/docs/concepts/connectors/install-storage-connector
       
      Also, HDFS reads and writes from/to GCS.
       
      Made sure java version matches and appropriate values are in classpath.
       
      Appreciate your time and effort.
      Thanks

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Haaris Haaris
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: