[HBASE-20244] NoSuchMethodException when retrieving private method decryptEncryptedDataEncryptionKey from DFSClient

• Type: Sub-task
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.0.0, 2.0.1
• Fix Version/s: 3.0.0-alpha-1, 2.1.0, 2.0.2
• Component/s: wal
• Hadoop Flags: Reviewed

Release Note

HDFS-12574 made an incompatible change to HdfsKMSUtil with different method signature.
This issue uses reflection to try known method signatures in order to work with hadoop releases with and without HDFS-12574

• Assignee: Ted Yu
• Reporter: Ted Yu
• Created: 21/Mar/18 14:51
• Updated: 23/Jun/22 19:29
• Resolved: 03/Jul/18 14:26

Description

I was running unit test against hadoop 3.0.1 RC and saw the following in test output:

ERROR [RS-EventLoopGroup-3-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(267): Couldn't properly initialize access to HDFS internals. Please update  your WAL Provider to not make use of the 'asyncfs' provider. See HBASE-16110 for more information.
java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo)
  at java.lang.Class.getDeclaredMethod(Class.java:2130)
  at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:232)
  at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.<clinit>(FanOutOneBlockAsyncDFSOutputSaslHelper.java:262)
  at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:661)
  at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:118)
  at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:720)
  at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$13.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:715)
  at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
  at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)
  at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)
  at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
  at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
  at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
  at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:306)
  at org.apache.hbase.thirdparty.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:341)
  at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
  at org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)

The private method was moved by HDFS-12574 to HdfsKMSUtil with different signature.

To accommodate the above method movement, it seems we need to call the following method of DFSClient :

  public KeyProvider getKeyProvider() throws IOException {

Since the new decryptEncryptedDataEncryptionKey method has this signature:

  static KeyVersion decryptEncryptedDataEncryptionKey(FileEncryptionInfo
        feInfo, KeyProvider keyProvider) throws IOException {

Issue Links

• is broken by: HDFS-12396 Webhdfs file system should get delegation token from kms provider.
• is broken by: HDFS-12574 Add CryptoInputStream to WebHdfsFileSystem read call.
• is related to: HBASE-20503 [AsyncFSWAL] Failed to get sync result after 300000 ms for txid=160912, WAL system stuck?