Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-77

Impala does not properly handle HDFS viewfs mounts (federated cluster) - Queries fail with "wrong filesystem" error

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 0.5
    • Fix Version/s: Impala 1.0
    • Component/s: None
    • Labels:
      None

      Description

      Impala is failing to read data on the HDFS viewFS file system mounts (federated cluster)

      <property>
      <name>fs.defaultFS</name>
      <value>viewfs://cluster24</value>
      </property>
      <property>
      <name>fs.viewfs.mounttable.cluster24.link./tmp</name>
      <value>hdfs://nameservice1/tmp</value>
      </property>
      <property>
      <name>fs.viewfs.mounttable.cluster24.link./user</name>
      <value>hdfs://nameservice1/user</value>
      </property>
      <property>
      <name>fs.viewfs.mounttable.cluster24.link./var</name>
      <value>hdfs://nameservice1/var</value>
      </property>
      
      Impala queries will fail to query this with
      Exception in thread "Thread-7" java.lang.IllegalArgumentException: Wrong FS: viewfs://cluster24/test-warehouse/foo2, expected: 
      at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:551)
      at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:166)
      at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:212)
      at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:75)
      Call to org.apache.hadoop.conf.FileSystem::open((Lorg/apache/hadoop/fs/Path;I)Lorg/apache/hadoop/fs/FSDataInputStream failed!
      

        Attachments

          Activity

            People

            • Assignee:
              alan@cloudera.com Alan Choi
              Reporter:
              lskuff Lenni Kuff
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: