[IMPALA-7809] test_concurrent_schema_change incompatible with Kudu 1.9

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 3.2.0
• Fix Version/s: Impala 3.2.0
• Component/s: Infrastructure
• Labels: broken-build
• Target Version: Impala 3.2.0
• Epic Color: ghx-label-4
• Assignee: Michael Brown
• Reporter: Michael Brown
• Created: 05/Nov/18 22:33
• Updated: 06/Nov/18 21:37
• Resolved: 06/Nov/18 21:37

Description

With Kudu 1.9, test_concurrent_schema_change incompatible fails because the message format is slightly different.

query_test/test_kudu.py:442: in test_concurrent_schema_change     assert "has fewer columns (1) than the SELECT / VALUES clause returns (2)" in msg \ E   assert ('has fewer columns (1) than the SELECT / VALUES clause returns (2)' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error: Invalid argument: Failed to write batch ...redacted.com:31201): Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n' or "(type: TINYINT) is not compatible with column 'col1' (type: STRING)" in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error: Invalid argument: Failed to write batch ...redacted.com:31201): Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n' or 'has fewer columns than expected.' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error: Invalid argument: Failed to write batch ...redacted.com:31201): Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n' or 'Column col1 has unexpected type.' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error: Invalid argument: Failed to write batch ...redacted.com:31201): Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n' or 'Client provided column col1[int64 NULLABLE] not present in tablet' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error: Invalid argument: Failed to write batch ...redacted.com:31201): Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n')
Stacktrace
query_test/test_kudu.py:442: in test_concurrent_schema_change
    assert "has fewer columns (1) than the SELECT / VALUES clause returns (2)" in msg \
E   assert ('has fewer columns (1) than the SELECT / VALUES clause returns (2)' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error: Invalid argument: Failed to write batch ...redacted.com:31201): Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n' or "(type: TINYINT) is not compatible with column 'col1' (type: STRING)" in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error: Invalid argument: Failed to write batch ...redacted.com:31201): Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n' or 'has fewer columns than expected.' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error: Invalid argument: Failed to write batch ...redacted.com:31201): Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n' or 'Column col1 has unexpected type.' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error: Invalid argument: Failed to write batch ...redacted.com:31201): Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n' or 'Client provided column col1[int64 NULLABLE] not present in tablet' in 'ImpalaBeeswaxException:\n Query aborted:Kudu error(s) reported, first error: Invalid argument: Failed to write batch ...redacted.com:31201): Client provided column col1 INT64 NULLABLE not present in tablet (1 of 2 similar)\n\n')

We see that the message returned includes Client provided column col1 INT64 NULLABLE not present in tablet and the code is looking for:

 437     for error in insert_thread.errors:
 438       msg = str(error)
 439       # The first two are AnalysisExceptions, the next two come from KuduTableSink::Open()
 440       # if the schema has changed since analysis, the last comes from the Kudu server if
 441       # the schema changes between KuduTableSink::Open() and when the write ops are sent.
 442       assert "has fewer columns (1) than the SELECT / VALUES clause returns (2)" in msg \
 443         or "(type: TINYINT) is not compatible with column 'col1' (type: STRING)" in msg \
 444         or "has fewer columns than expected." in msg \
 445         or "Column col1 has unexpected type." in msg \
 446         or "Client provided column col1[int64 NULLABLE] not present in tablet" in msg