[IMPALA-8091] minicluster kudu failure: Cannot initialize clock: failed to wait for clock sync

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 3.2.0
• Fix Version/s: Impala 3.2.0
• Component/s: Infrastructure
• Labels: broken-build
• Epic Color: ghx-label-4
• Assignee: Michael Brown
• Reporter: David Knupp
• Created: 17/Jan/19 22:26
• Updated: 08/Feb/19 16:14
• Resolved: 06/Feb/19 19:43

Description

Console log:

19:10:04 19:10:04 Error executing impala SQL: /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql See: /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-kudu-none-none.sql.log
19:10:04 Background task Loading functional-query data (pid 4831) failed.
19:10:04 Background task Loading TPC-H data (pid 4832) failed.
19:10:04 ERROR in /data/jenkins/workspace/impala-asf-master-exhaustive-release/repos/Impala/testdata/bin/create-load-data.sh at line 85: CM_HOST=${2-}
[...]
19:10:05 2019-01-16 19:10:05,461 - archive_core_dumps - INFO - Found core files: ['./core.1547693849.30799.kudu-master', './core.1547693849.30783.kudu-tserver', './core.1547693849.30767.kudu-tserver', './core.1547693849.30808.kudu-tserver']
19:10:05 2019-01-16 19:10:05,649 - archive_core_dumps - INFO - [New LWP 30799]
19:10:05 [New LWP 30834]
19:10:05 [New LWP 30835]
19:10:05 [New LWP 30838]
19:10:05 [New LWP 30837]
19:10:05 [New LWP 30836]
19:10:05 Core was generated by `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
19:10:05 Program terminated with signal SIGABRT, Aborted.
19:10:05 #0  0x00007f9a2cc611f7 in ?? ()
19:10:05 
19:10:05 2019-01-16 19:10:05,650 - archive_core_dumps - INFO - Found binary path through GDB: /data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c
19:10:05 2019-01-16 19:10:05,893 - archive_core_dumps - WARNING - Failed to determine binary because multiple candidate binaries were found and none of their paths contained 'latest' to disambiguate:
19:10:05 Core:./core.1547693849.30799.kudu-master
19:10:05 Binaries:['./testdata/cluster/node_templates/common/etc/init.d/kudu-master', './testdata/cluster/cdh6/node-1/etc/init.d/kudu-master']
19:10:05 
19:10:05 2019-01-16 19:10:05,917 - archive_core_dumps - INFO - [New LWP 30783]
19:10:05 [New LWP 30810]
19:10:05 [New LWP 30812]
19:10:05 [New LWP 30811]
19:10:05 [New LWP 30824]
19:10:05 [New LWP 30820]
19:10:05 Core was generated by `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
19:10:05 Program terminated with signal SIGABRT, Aborted.
19:10:05 #0  0x00007f0b81fb11f7 in ?? ()

Backtraces:

CORE: ./core.1547693849.30799.kudu-master
BINARY: ./be/build/latest/service/impalad
Core was generated by `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007f9a2cc611f7 in ?? ()
#0  0x00007f9a2cc611f7 in ?? ()
#1  0x00007f9a2cc628e8 in ?? ()
#2  0x0000000000000020 in ?? ()
#3  0x0000000000000000 in ?? ()

CORE: ./core.1547693849.30783.kudu-tserver
BINARY: ./be/build/latest/service/impalad
Core was generated by `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007f0b81fb11f7 in ?? ()
#0  0x00007f0b81fb11f7 in ?? ()
#1  0x00007f0b81fb28e8 in ?? ()
#2  0x0000000000000020 in ?? ()
#3  0x0000000000000000 in ?? ()

CORE: ./core.1547693849.30767.kudu-tserver
BINARY: ./be/build/latest/service/impalad
Core was generated by `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007f60b1e2f1f7 in ?? ()
#0  0x00007f60b1e2f1f7 in ?? ()
#1  0x00007f60b1e308e8 in ?? ()
#2  0x0000000000000020 in ?? ()
#3  0x0000000000000000 in ?? ()

CORE: ./core.1547693849.30808.kudu-tserver
BINARY: ./be/build/latest/service/impalad
Core was generated by `/data/jenkins/workspace/impala-asf-master-exhaustive-release/Impala-Toolchain/c'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007fa3cb5591f7 in ?? ()
#0  0x00007fa3cb5591f7 in ?? ()
#1  0x00007fa3cb55a8e8 in ?? ()
#2  0x0000000000000020 in ?? ()
#3  0x0000000000000000 in ?? ()

Issue Links

• is duplicated by: IMPALA-8120 Build Failure: Undetermined data load error for tpcds for Kudu
• is duplicated by: IMPALA-8128 Build failure: Kudu functional data load - Kudu crashed
• is related to: IMPALA-8128 Build failure: Kudu functional data load - Kudu crashed