Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-20603

"Wrong FS" error when inserting to partition after changing table location filesystem

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Inserting into an existing partition, after changing a table's location to point to a different HDFS filesystem:

         query += "CREATE TABLE test_managed_tbl (id int, name string, dept string) PARTITIONED BY (year int);\n"
          query += "INSERT INTO test_managed_tbl PARTITION (year=2016) VALUES (8,'Henry','CSE');\n"
          query += "ALTER TABLE test_managed_tbl ADD PARTITION (year=2017);\n"
          query += "ALTER TABLE test_managed_tbl SET LOCATION 
                                        'hdfs://ns2/warehouse/tablespace/managed/hive/test_managed_tbl'"
          query += "INSERT INTO test_managed_tbl PARTITION (year=2017) VALUES (9,'Harris','CSE');\n"
      

      Results in the following error:

      java.lang.IllegalArgumentException: Wrong FS: hdfs://ns1/warehouse/tablespace/managed/hive/test_managed_tbl/year=2017, expected: hdfs://ns2
              at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:781)
              at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:240)
              at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1583)
              at org.apache.hadoop.hdfs.DistributedFileSystem$29.doCall(DistributedFileSystem.java:1580)
              at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
              at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1595)
              at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1734)
              at org.apache.hadoop.hive.ql.metadata.Hive.copyFiles(Hive.java:4141)
              at org.apache.hadoop.hive.ql.metadata.Hive.loadPartition(Hive.java:1966)
              at org.apache.hadoop.hive.ql.exec.MoveTask.handleStaticParts(MoveTask.java:477)
              at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:397)
              at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:210)
              at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97)
              at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2701)
              at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2372)
              at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2048)
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1746)
              at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1740)
      

        Attachments

        1. HIVE-20603.1.patch
          15 kB
          Jason Dere
        2. HIVE-20603.2.patch
          17 kB
          Jason Dere
        3. HIVE-20603.3.patch
          17 kB
          Jason Dere

          Issue Links

            Activity

              People

              • Assignee:
                jdere Jason Dere
                Reporter:
                jdere Jason Dere
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: