Details
-
Bug
-
Status: Resolved
-
Critical
-
Resolution: Fixed
-
None
-
None
-
Reviewed
Description
This is a code sniff from the discussion of HBASE-27073
public static void main(String[] args) throws Exception { CompressionContext ctx = new CompressionContext(LRUDictionary.class, false, false, true, Compression.Algorithm.GZ); ValueCompressor compressor = ctx.getValueCompressor(); byte[] compressed = compressor.compress(new byte[0], 0, 0); System.out.println("compressed length: " + compressed.length); ByteArrayInputStream bis = new ByteArrayInputStream(compressed); int read = compressor.decompress(bis, compressed.length, new byte[0], 0, 0); System.out.println("read length: " + read); System.out.println("position: " + (compressed.length - bis.available()));
And the output is
compressed length: 20 read length: 0 position: 0
So it turns out that, when compressing, an empty array will still generate some output bytes but while reading, we will skip reading anything if we find the output length is zero, so next time when we read from the stream, we will start at a wrong position...
Attachments
Issue Links
- is related to
-
HBASE-27073 TestReplicationValueCompressedWAL.testMultiplePuts is flaky
- Resolved
- links to
1.
|
Optimize writes of zero length values in compressed WALs | Open | Unassigned |