Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-8131 Improve Database support
  3. SPARK-8714

Refresh table not working with non-default databases

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 1.5.0
    • SQL
    • None

    Description

      I created an external table with sqlContext.createExternalTable from parquet data in S3. After that, I placed a new partition (directory with some parquet files) in S3. Then I did a sql refresh table. It worked great.

      I created a separate cluster and changed the database from default to my database. I repeated the above process for the new database. Now the createExternalTable works fine but the refresh table doesn't work. The first time I got an exception stack trace but the next time I don't see any error but when I do a count on the # of rows, the new data is not reflected.

      Attachments

        Activity

          People

            lian cheng Cheng Lian
            prakash573 Prakash Chockalingam
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: