[FLINK-30790] [Connectors/Jdbc] Refactor of testings

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: jdbc-3.2.0
• Component/s: Connectors / JDBC
• Labels: pull-request-available
• Assignee: João Boto
• Reporter: João Boto
• Created: 25/Jan/23 16:27
• Updated: 24/May/23 09:42
• Resolved: 24/May/23 09:42

Description

This is one attempt to get better results executing tests on jdbc connector..

In current master branch it takes like 15m to execute all tests on CI (it will vary from 10m to 20m)

 

[INFO] Reactor Summary for Flink : Connectors : JDBC : Parent 3.1-SNAPSHOT:
[INFO] 
[INFO] Flink : Connectors : JDBC : Parent ................. SUCCESS [ 22.019 s]
[INFO] Flink : Connectors : JDBC .......................... SUCCESS [13:45 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  14:07 min
[INFO] Finished at: 2022-12-23T09:22:46Z 

 

 

The main problems that we see on testings are:

• Parameterized tests that setup database containers (this tends to be more time consuming as another databases are added)
• Creation of containers by class (this is fine as we want test to be independents, but the database could be setup 1time and cleaned at end of each class)
• No easy way to extend a test to another database, at the end we copy a lot of code
• A lot of code for create and populate tables for testing. We have JdbcTestBase that use JdbcTestFixture to create a kind of 'Book Store', but this is used in a lot of test without the use of all tables that are implemented on store. 

 

 

Issue Links

• fixes: FLINK-31770 OracleExactlyOnceSinkE2eTest.testInsert fails for JDBC connector
• links to: GitHub Pull Request #18
• links to: GitHub Pull Request #22