[FLINK-34420] Various YARN tests fail after failing to download hadoop.tar.gz

• Type: Technical Debt
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.19.0, 1.18.2, 1.20.0
• Fix Version/s: 1.19.0, 1.17.3, 1.18.2
• Component/s: Tests
• Assignee: Roman Khachatryan
• Reporter: Roman Khachatryan
• Created: 10/Feb/24 08:47
• Updated: 27/Feb/24 14:16
• Resolved: 11/Feb/24 12:32

Description

https://dev.azure.com/khachatryanroman/flink/_build/results?buildId=1702&view=logs&j=0e31ee24-31a6-528c-a4bf-45cde9b2a14e&t=696bc156-f753-5888-468e-42d78df39222&l=11334

https://dev.azure.com/khachatryanroman/flink/_build/results?buildId=1700&view=logs&j=bbbd0720-137e-5f59-95a5-b5d332f196d3&t=4769aa47-e87b-5ecd-1fb2-14d52396866d&l=9937

```
2024-02-09T19:21:57.8947690Z Feb 09 19:21:57 Pre-downloading Hadoop tarball
2024-02-09T19:21:57.9250518Z % Total % Received % Xferd Average Speed Time Time Time Current
2024-02-09T19:21:57.9250844Z Dload Upload Total Spent Left Speed
2024-02-09T19:21:57.9251910Z
2024-02-09T19:21:58.0005684Z 0 0 0 0 0 0 0 0 -::- -::- -::- 0
2024-02-09T19:21:58.0006583Z 100 288 100 288 0 0 3789 0 -::- -::- -::- 3789
```
which is way too small - meaning we got redirection (https://dev.azure.com/khachatryanroman/flink/_build/results?buildId=1700&view=logs&j=bbbd0720-137e-5f59-95a5-b5d332f196d3&t=4769aa47-e87b-5ecd-1fb2-14d52396866d&l=8056 )

Later, it can't be unpacked:
https://dev.azure.com/khachatryanroman/flink/_build/results?buildId=1700&view=logs&j=bbbd0720-137e-5f59-95a5-b5d332f196d3&t=4769aa47-e87b-5ecd-1fb2-14d52396866d&l=9657
```
#11 [ 7/28] COPY hadoop.tar.gz /tmp/hadoop.tar.gz
#11 CACHED

#12 [ 8/28] RUN set -x && mkdir -p /usr/local/hadoop && tar -xf /tmp/hadoop.tar.gz --strip-components=1 -C /usr/local/hadoop && rm /tmp/hadoop.tar.gz*
#12 0.175 + mkdir -p /usr/local/hadoop
#12 0.177 + tar -xf /tmp/hadoop.tar.gz --strip-components=1 -C /usr/local/hadoop
#12 0.178 tar: This does not look like a tar archive
#12 0.179
#12 0.179 gzip: stdin: not in gzip format
#12 0.179 tar: Child returned status 1
#12 0.179 tar: Error is not recoverable: exiting now

```