Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-29174

LOCAL is not supported in INSERT OVERWRITE DIRECTORY to data source

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.0
    • 3.1.0
    • SQL
    • None

    Description

      using does not work for insert overwrite when in local  but works when insert overwrite in HDFS directory

      
      0: jdbc:hive2://10.18.18.214:23040/default> insert overwrite directory '/user/trash2/' using parquet select * from trash1 a where a.country='PAK';
      
      +---------+--+
      | Result  |
      +---------+--+
      +---------+--+
      
      No rows selected (0.448 seconds)
      
      0: jdbc:hive2://10.18.18.214:23040/default> insert overwrite local directory '/opt/trash2/' using parquet select * from trash1 a where a.country='PAK';
      
      Error: org.apache.spark.sql.catalyst.parser.ParseException:
      
      LOCAL is not supported in INSERT OVERWRITE DIRECTORY to data source(line 1, pos 0)
      
       
      
      == SQL ==
      
      insert overwrite local directory '/opt/trash2/' using parquet select * from trash1 a where a.country='PAK'
      
      ^^^ (state=,code=0)
      
      0: jdbc:hive2://10.18.18.214:23040/default> insert overwrite local directory '/opt/trash2/' stored as parquet select * from trash1 a where a.country='PAK';
      
      +---------+--+
      | Result  |
      +---------+--+
      | | |
      

       

      Attachments

        Issue Links

          Activity

            People

              ajithshetty Ajith S
              abhishek.akg ABHISHEK KUMAR GUPTA
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: