Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-6580

Cannot load Kudu functional data on localfs build

    Details

      Description

      Data loading fails here:

      .
      08:15:58 Skipping table: functional_kudu.kite_required_fields, 'only' constraint for format did not include this table.
      08:15:58 Skipping table: functional_kudu.bad_column_metadata, 'only' constraint for format did not include this table.
      08:15:58 Skipping table: functional_kudu.bad_serde, 'only' constraint for format did not include this table.
      08:15:58 Skipping table: functional_kudu.rcfile_lazy_binary_serde, 'only' consmkdir: `/test-warehouse/decimal_tbl/d6=1': Input/output error
      08:15:58 traint for format did not include this table.
      08:15:58 Traceback (most recent call last):
      08:15:58   File "/data/jenkins/workspace/impala-asf-master-core-local/repos/Impala/testdata/bin/generate-schema-statements.py", line 763, in <module>
      08:15:58     test_vectors, sections, include_constraints, exclude_constraints, only_constraints)
      08:15:58   File "/data/jenkins/workspace/impala-asf-master-core-local/repos/Impala/testdata/bin/generate-schema-statements.py", line 534, in generate_statements
      08:15:58     load = eval_section(section['LOAD'])
      08:15:58   File "/data/jenkins/workspace/impala-asf-master-core-local/repos/Impala/testdata/bin/generate-schema-statements.py", line 473, in eval_section
      08:15:58     assert p.returncode == 0
      08:15:58 AssertionError
      08:15:58 Error generating schema statements for workload: functional-query
      08:15:58 Background task Loading Kudu functional (pid 26145) failed.
      

      The problem is that the "LOAD" section is being executed for Kudu

      ====
      ---- DATASET
      functional
      ---- BASE_TABLE_NAME
      decimal_tbl
      ---- COLUMNS
      d1 DECIMAL
      d2 DECIMAL(10, 0)
      d3 DECIMAL(20, 10)
      d4 DECIMAL(38, 38)
      d5 DECIMAL(10, 5)
      ---- PARTITION_COLUMNS
      d6 DECIMAL(9, 0)
      ---- ALTER
      ALTER TABLE {table_name} ADD IF NOT EXISTS PARTITION(d6=1);
      ---- ROW_FORMAT
      delimited fields terminated by ','
      ---- LOAD
      `hadoop fs -mkdir -p /test-warehouse/decimal_tbl/d6=1 && hadoop fs -put -f \
      ${IMPALA_HOME}/testdata/data/decimal_tbl.txt /test-warehouse/decimal_tbl/d6=1/
      ---- DEPENDENT_LOAD
      INSERT OVERWRITE TABLE {db_name}{db_suffix}.{table_name} partition(d6)
      select * from functional.{table_name};
      ---- CREATE_KUDU
      DROP TABLE IF EXISTS {db_name}{db_suffix}.{table_name};
      CREATE TABLE {db_name}{db_suffix}.{table_name} (
        d1 DECIMAL,
        d2 DECIMAL(10, 0),
        d3 DECIMAL(20, 10),
        d4 DECIMAL(38, 38),
        d5 DECIMAL(10, 5),
        d6 DECIMAL(9, 0),
        PRIMARY KEY (d1, d2, d3, d4, d5, d6)
      )
      PARTITION BY HASH PARTITIONS 3
      STORED AS KUDU;
      ---- DEPENDENT_LOAD_KUDU
      INSERT into TABLE {db_name}{db_suffix}.{table_name}
      SELECT d1, d2, d3, d4, d5, d6
      FROM {db_name}.{table_name};
      
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                joemcdonnell Joe McDonnell
                Reporter:
                tarmstrong Tim Armstrong
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: