[SPARK-37011] Upgrade flake8 to 3.9.0 or above in Jenkins

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.0
• Fix Version/s: 3.3.0
• Component/s: PySpark
• Assignee: Shane Knapp
• Reporter: Takuya Ueshin
• Created: 15/Oct/21 00:50
• Updated: 25/Oct/21 20:05
• Resolved: 25/Oct/21 19:55

Description

In flake8 < 3.9.0, F401 error occurs for imports when the imported identities are used in a bound argument in TypeVar(..., bound="XXX").

For example:

if TYPE_CHECKING:
    from pyspark.pandas.base import IndexOpsMixin

IndexOpsLike = TypeVar("IndexOpsLike", bound="IndexOpsMixin")

Since this behavior is fixed In flake8 >= 3.9.0, we should upgrade the flake8 installed in Jenkins to 3.9.0 or above.
And also we might update the MINIMUM_FLAKE8 in the lint-python from 3.8.0 to 3.9.0.

Issue Links

• is a clone of: SPARK-34943 Upgrade flake8 to 3.8.0 or above in Jenkins
• links to: [Github] Pull Request #34384 (shaneknapp)
• links to: [Github] Pull Request #34385 (ueshin)