Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-1714

Trying to cache a table which has its data in s3 results in an IllegalArgumentException

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 2.2
    • Impala 2.5.0
    • None

    Description

      Table's in s3:

      Query: show create table functional.alltypestiny
      "CREATE EXTERNAL TABLE functional.alltypestiny (
        id INT, 
        bool_col BOOLEAN, 
        tinyint_col TINYINT, 
        smallint_col SMALLINT, 
        int_col INT, 
        bigint_col BIGINT, 
        float_col FLOAT, 
        double_col DOUBLE, 
        date_string_col STRING, 
        string_col STRING, 
        timestamp_col TIMESTAMP
      )
      PARTITIONED BY (
        year INT, 
        month INT
      )
      ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' ESCAPED BY '\\'
      WITH SERDEPROPERTIES ('field.delim'=',', 'serialization.format'=',', 'escape.delim'='\\')
      STORED AS TEXTFILE
      LOCATION 's3a://<s3_bucket>/test-warehouse/alltypestiny'
      TBLPROPERTIES ('STATS_GENERATED_VIA_STATS_TASK'='true', 'transient_lastDdlTime'='1422275382', 'numRows'='8')"
      Fetched 1 row(s) in 0.01s
      

      When I try to cache the table using an existing cache pool it gives me an IllegalArgumentException. We want to be able to have tables that exist across file systems, so this is not good.

      [ishaan-impala-ec2.vpc.cloudera.com:21000] > alter table functional.alltypestiny set cached in "testPool";
      Query: alter table functional.alltypestiny set cached in "testPool"
      ERROR: IllegalArgumentException: Wrong FS: s3a://<s3_bucket>/test-warehouse/alltypestiny, expected: hdfs://localhost:20500
      

      Attachments

        Activity

          People

            kwho Michael Ho
            ishaan Ishaan Joshi
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: