[FLINK-15311] Lz4BlockCompressionFactory should use native compressor instead of java unsafe

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.10.0
• Component/s: Runtime / Network
• Labels: pull-request-available
• Assignee: Yingjie Cao
• Reporter: Jingsong Lee
• Created: 18/Dec/19 08:41
• Updated: 20/Dec/19 15:36
• Resolved: 20/Dec/19 15:36

Description

According to:

https://lz4.github.io/lz4-java/1.7.0/lz4-compression-benchmark/

Java java unsafe compressor has lower performance than native lz4 compressor.

After FLINK-14845 , we use lz4 compression for shuffler.

In testing, I found shuffle using java unsafe compressor.

Issue Links

• links to: GitHub Pull Request #10636
• links to: GitHub Pull Request #10639