Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-9822

Impala does not notify user that row format delimited fields is only logical when using STORED AS TEXTFILE

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: Impala 3.4.0
    • Fix Version/s: None
    • Component/s: Frontend
    • Epic Color:
      ghx-label-11

      Description

      When creating a table with added "ROW FORMAT DELIMITED FIELDS", Impala does not alert the user that this is only logical when using STORED AS TEXTFILE.

      You only discover that you made a mistake after trying to run a select from the table.

       Table creation:

      [adunai-1.adunai.root.hwx.site:21000] default> CREATE EXTERNAL TABLE sales_fact_1997(product_id INT,time_id INT,customer_id INT,promotion_id INT,store_id INT,store_sales DECIMAL(10,4),store_cost DECIMAL(10,4),unit_sales DECIMAL(10,4))
       > row format delimited fields terminated by '\011' STORED AS PARQUET
       > location '/user/impala/mondrian/sales_fact_1997';
      Query: CREATE EXTERNAL TABLE sales_fact_1997(product_id INT,time_id INT,customer_id INT,promotion_id INT,store_id INT,store_sales DECIMAL(10,4),store_cost DECIMAL(10,4),unit_sales DECIMAL(10,4))row format delimited fields terminated by '\011' STORED AS PARQUET location '/user/impala/mondrian/sales_fact_1997'
       
      +-------------------------+
      | summary |
      +-------------------------+
      | Table has been created. |
      +-------------------------+
      Fetched 1 row(s) in 0.10s
      

       

      Select: 

      [adunai-1.adunai.root.hwx.site:21000] mondrian> select count(*) from agg_c_10_sales_fact_1997;
      Query: select count(*) from agg_c_10_sales_fact_1997
      Query submitted at: 2020-06-03 11:55:06 (Coordinator: http://adunai-1.adunai.root.hwx.site:25000)
      Query progress can be monitored at: http://adunai-1.adunai.root.hwx.site:25000/query_plan?query_id=d547fafd0162da4e:872a95c100000000
      ERROR: File 'hdfs://adunai-2.adunai.root.hwx.site:8020/user/impala/mondrian/agg_c_10_sales_fact_1997/agg_c_10_sales_fact_1997.tsv' has an invalid Parquet version number: 717. Please check that it is a valid Parquet file. This error can also occur due to stale metadata. If you believe this is a valid Parquet file, try running "refresh mondrian.agg_c_10_sales_fact_1997".

        Attachments

          Activity

            People

            • Assignee:
              shikha.asrani Shikha Asrani
              Reporter:
              adunai Alexandra Dunai
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: