[FLINK-24336] PyFlink TableEnvironment executes the SQL randomly MalformedURLException with the configuration for 'pipeline.classpaths'

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.0, 1.13.2, 1.14.1
• Fix Version/s: 1.13.3, 1.12.8, 1.14.3, 1.15.0
• Component/s: API / Python
• Labels: pull-request-available
• Assignee: Xin Yu
• Reporter: Nicholas Jiang
• Created: 18/Sep/21 08:58
• Updated: 15/Dec/21 01:44
• Resolved: 24/Sep/21 01:45

Description

When I run flink client to submit a python based workflow, I got the MalformedURLException like this:

https://gist.github.com/is/faabafc7f8750f3f3161fbb6517ed6ff

After some debug work, I found the problem is related with
TableEvneriontment.execute_sql. The root cause is
TableEvenriontment._add_jars_to_j_env_config in pyflink/table/TableEnverionment.py.

```
if j_configuration.containsKey(config_key):
for url in j_configuration.getString(config_key, "").split(";"):
jar_urls_set.add(url)
```

In our case, pipeline.classpaths was set by empty list value
from FromProgramOption, so the upper code block will
introduce a empty string ("") into pipeline.classpaths, for example
"a.jar;b.jar;;c.jar", and it will cause the according exception.

Another problem, the order of string set in python is not
determinate, so ";".join(jar_urls_set) does NOT keep the
classpaths order. The list is more suiteable in this case.

Issue Links

• is related to: FLINK-24390 Python 'build_wheels mac' fails on azure
• links to: GitHub Pull Request #17322