[IMPALA-2913] Impalad fails to load tables in test runs

• Type: Bug
• Status: Reopened
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Impala 2.5.0
• Fix Version/s: Impala 2.5.0
• Component/s: Catalog, Frontend
• Target Version: Impala 2.5.0
• Assignee: Vuk Ercegovac
• Reporter: Henry Robinson
• Created: 30/Jan/16 22:30
• Updated: 17/Sep/18 14:00

Description

Build: http://sandbox.jenkins.cloudera.com/job/impala-master-cdh5-trunk/1714/

impalad

I0130 10:12:40.874454 24955 Frontend.java:864] analyze query drop database if exists `test_parquet_list_encodings_793898492` cascade
I0130 10:13:41.530505 24944 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:13:41.532280 24944 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.customer_demographics
I0130 10:13:43.693908 24948 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:13:43.697772 24948 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.household_demographics, tpcds_parquet.customer
I0130 10:13:51.970826 24928 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:13:51.971350 24928 Frontend.java:808] Requesting prioritized load of table(s): functional_avro_snap.alltypestiny
I0130 10:14:10.715591 24925 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:14:10.718114 24925 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.customer, tpcds_parquet.promotion
I0130 10:15:41.542430 24944 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:15:41.544220 24944 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.customer_demographics
I0130 10:15:43.707329 24948 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:15:43.711132 24948 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.household_demographics, tpcds_parquet.customer
I0130 10:15:51.980530 24928 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:15:51.981075 24928 Frontend.java:808] Requesting prioritized load of table(s): functional_avro_snap.alltypestiny
I0130 10:16:10.728305 24925 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:16:10.730826 24925 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.customer, tpcds_parquet.promotion
I0130 10:17:41.554452 24944 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:17:41.556248 24944 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.customer_demographics
I0130 10:17:43.720675 24948 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:17:43.724488 24948 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.household_demographics, tpcds_parquet.customer
I0130 10:17:51.990448 24928 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:17:51.990978 24928 Frontend.java:808] Requesting prioritized load of table(s): functional_avro_snap.alltypestiny
I0130 10:18:10.740888 24925 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:18:10.743422 24925 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.customer, tpcds_parquet.promotion
I0130 10:19:41.566427 24944 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:19:41.568217 24944 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.customer_demographics
I0130 10:19:43.733989 24948 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:19:43.737788 24948 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.household_demographics, tpcds_parquet.customer
I0130 10:19:52.000059 24928 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:19:52.000630 24928 Frontend.java:808] Requesting prioritized load of table(s): functional_avro_snap.alltypestiny
I0130 10:20:10.753553 24925 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:20:10.756093 24925 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.customer, tpcds_parquet.promotion
I0130 10:21:41.578438 24944 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:21:41.580238 24944 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.customer_demographics
I0130 10:21:43.747339 24948 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:21:43.751142 24948 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.household_demographics, tpcds_parquet.customer
I0130 10:21:52.009795 24928 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:21:52.010337 24928 Frontend.java:808] Requesting prioritized load of table(s): functional_avro_snap.alltypestiny
I0130 10:22:10.766197 24925 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:22:10.768699 24925 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.customer, tpcds_parquet.promotion
I0130 10:23:41.590198 24944 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:23:41.591982 24944 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.customer_demographics
I0130 10:23:43.760777 24948 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:23:43.764662 24948 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.household_demographics, tpcds_parquet.customer
I0130 10:23:52.019451 24928 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:23:52.020020 24928 Frontend.java:808] Requesting prioritized load of table(s): functional_avro_snap.alltypestiny
I0130 10:24:10.778784 24925 Frontend.java:883] Missing tables were not received in 120000ms. Load request will be retried.
I0130 10:24:10.781328 24925 Frontend.java:808] Requesting prioritized load of table(s): tpcds_parquet.customer, tpcds_parquet.promotion

... and so on until 4 hours have elapsed and the build gets aborted.

In the catalog:

catalogd

I0130 10:12:10.705793 24706 TableLoadingMgr.java:278] Loading next table. Remaining items in queue: 0
I0130 10:12:40.875004 29633 CatalogOpExecutor.java:1031] Dropping database test_parquet_list_encodings_793898492
I0130 10:12:40.875083 29633 MetaStoreClientPool.java:55] Creating MetaStoreClient. Pool Size = 0
I0130 10:12:40.875221 29633 HiveMetaStoreClient.java:376] Trying to connect to metastore with URI thrift://localhost:9083
I0130 10:12:40.875553 29633 HiveMetaStoreClient.java:421] Opened a connection to metastore, current connections: 136
I0130 10:12:40.876093 29633 HiveMetaStoreClient.java:473] Connected to metastore.
I0130 10:13:41.532707 24701 TableLoadingMgr.java:278] Loading next table. Remaining items in queue: 0
I0130 10:13:43.698223 24709 TableLoadingMgr.java:278] Loading next table. Remaining items in queue: 1
I0130 10:13:43.698299 24696 TableLoadingMgr.java:278] Loading next table. Remaining items in queue: 0
I0130 10:13:51.971761 24700 TableLoadingMgr.java:278] Loading next table. Remaining items in queue: 0
I0130 10:14:10.718595 24694 TableLoadingMgr.java:278] Loading next table. Remaining items in queue: 1
I0130 10:14:10.718680 24707 TableLoadingMgr.java:278] Loading next table. Remaining items in queue: 0
I0130 10:15:41.544662 24708 TableLoadingMgr.java:278] Loading next table. Remaining items in queue: 0
I0130 10:15:43.711565 24699 TableLoadingMgr.java:278] Loading next table. Remaining items in queue: 1
I0130 10:15:43.711632 24705 TableLoadingMgr.java:278] Loading next table. Remaining items in queue: 0
I0130 10:15:51.981508 24697 TableLoadingMgr.java:278] Loading next table. Remaining items in queue: 0

.... nothing obvious.