[IMPALA-3679] Zero copy reads don't work with encryption zones

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: Impala 2.6.0
• Component/s: Backend
• Assignee: Bharath Vissapragada
• Reporter: Bharath Vissapragada
• Created: 06/Jun/16 08:32
• Updated: 30/Mar/18 16:54
• Resolved: 07/Jun/16 05:33

Description

Zero-copy reads do not work with files from encrypted zones. It fails with the following error.

java.nio.ReadOnlyBufferException
	at java.nio.DirectByteBufferR.put(DirectByteBufferR.java:344)
	at org.apache.hadoop.crypto.CryptoInputStream.decrypt(CryptoInputStream.java:533)
	at org.apache.hadoop.crypto.CryptoInputStream.read(CryptoInputStream.java:595)
	at org.apache.hadoop.fs.FSDataInputStream.read(FSDataInputStream.java:188)

This causes the ReadFromCache() call to fail and Impala switches to the normal read path using Read(). This causes a slight performance hit.

Workaround: Don't use hdfs caching on files from encrypted zones.

This error can even be reproduced using a standalone libhdfs client program to read a cached file content using zcr. I've reached out to atm and others to confirm that its a hadoop issue. Created this Impala jira just as a placeholder incase we need to update our libhdfs library.

Issue Links

• breaks: IMPALA-5593 Enable HDFS caching for impala when using encryption zones.