[BEAM-9578] Enumerating artifacts is too expensive in Java

• Type: Bug
• Status: Resolved
• Priority: P1
• Resolution: Fixed
• Fix Version/s: 2.21.0
• Component/s: sdk-java-core
• Labels: Done portability
• Assignee: Heejong Lee
• Reporter: Luke Cwik
• Created: 24/Mar/20 02:00
• Updated: 06/Oct/20 21:57
• Resolved: 08/Apr/20 21:08

Description

There are a lot of places (e.g. ParDoTranslation#getParDoPayload) which effectively call Environments#createOrGetDefaultEnvironment which causes artifacts to be computed.

This leads to zipping directories for non-jar dependencies.

Similar problems may exist for Python/Go.

Issue Links

• causes: BEAM-9534 Flink testParDoRequiresStableInput timed out
• is caused by: BEAM-9056 Staging artifacts from environment
• links to: GitHub Pull Request #11205
• links to: PR #10621