Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-20603

"Wrong FS" error when inserting to partition after changing table location filesystem

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 4.0.0-alpha-1
    • None
    • None

    Description

      Inserting into an existing partition, after changing a table's location to point to a different HDFS filesystem:

         query += "CREATE TABLE test_managed_tbl (id int, name string, dept string) PARTITIONED BY (year int);\n"
          query += "INSERT INTO test_managed_tbl PARTITION (year=2016) VALUES (8,'Henry','CSE');\n"
          query += "ALTER TABLE test_managed_tbl ADD PARTITION (year=2017);\n"
          query += "ALTER TABLE test_managed_tbl SET LOCATION 
                                        'hdfs://ns2/warehouse/tablespace/managed/hive/test_managed_tbl'"
          query += "INSERT INTO test_managed_tbl PARTITION (year=2017) VALUES (9,'Harris','CSE');\n"
      

      Results in the following error:

      java.lang.IllegalArgumentException: Wrong FS: hdfs://ns1/warehouse/tablespace/managed/hive/test_managed_tbl/year=2017, expected: hdfs://ns2
              at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:781)
              at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:240)
              at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1583)
              at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1580)
              at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
              at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1595)
              at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1734)
              at org.apache.hadoop.hive.ql.metadata.Hive.copyFiles(Hive.java:4141)
              at org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1966)
              at org.apache.hadoop.hive.ql.exec.MoveTask.handleStaticParts(MoveTask.java:477)
              at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:397)
              at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:210)
              at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97)
              at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2701)
              at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2372)
              at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2048)
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1746)
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1740)
      

      Attachments

        1. HIVE-20603.1.patch
          15 kB
          Jason Dere
        2. HIVE-20603.2.patch
          17 kB
          Jason Dere
        3. HIVE-20603.3.patch
          17 kB
          Jason Dere

        Issue Links

          Activity

            People

              jdere Jason Dere
              jdere Jason Dere
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: