Details
-
Task
-
Status: Closed
-
Blocker
-
Resolution: Done
-
0.9.0
Description
Trying to upgrade HBase to 2.3.3 but ran into several issues.
According to the Hadoop version support matrix: http://hbase.apache.org/book.html#hadoop, also need to upgrade Hadoop to 2.8.5+.
There are several API conflicts between HBase 2.2.3 and HBase 1.2.3, we need to resolve this first. After resolving conflicts, I am able to compile it but then I ran into a tricky jetty version issue during the testing:
[ERROR] TestHBaseIndex.testDelete() Time elapsed: 4.705 s <<< ERROR! java.lang.NoSuchMethodError: org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V [ERROR] TestHBaseIndex.testSimpleTagLocationAndUpdate() Time elapsed: 0.174 s <<< ERROR! java.lang.NoSuchMethodError: org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V [ERROR] TestHBaseIndex.testSimpleTagLocationAndUpdateWithRollback() Time elapsed: 0.076 s <<< ERROR! java.lang.NoSuchMethodError: org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V [ERROR] TestHBaseIndex.testSmallBatchSize() Time elapsed: 0.122 s <<< ERROR! java.lang.NoSuchMethodError: org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V [ERROR] TestHBaseIndex.testTagLocationAndDuplicateUpdate() Time elapsed: 0.16 s <<< ERROR! java.lang.NoSuchMethodError: org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V [ERROR] TestHBaseIndex.testTotalGetsBatching() Time elapsed: 1.771 s <<< ERROR! java.lang.NoSuchMethodError: org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V [ERROR] TestHBaseIndex.testTotalPutsBatching() Time elapsed: 0.082 s <<< ERROR! java.lang.NoSuchMethodError: org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V 34206 [Thread-260] WARN org.apache.hadoop.hdfs.server.datanode.DirectoryScanner - DirectoryScanner: shutdown has been called 34240 [BP-1058834949-10.0.0.2-1597189606506 heartbeating to localhost/127.0.0.1:55924] WARN org.apache.hadoop.hdfs.server.datanode.IncrementalBlockReportManager - IncrementalBlockReportManager interrupted 34240 [BP-1058834949-10.0.0.2-1597189606506 heartbeating to localhost/127.0.0.1:55924] WARN org.apache.hadoop.hdfs.server.datanode.DataNode - Ending block pool service for: Block pool BP-1058834949-10.0.0.2-1597189606506 (Datanode Uuid cb7bd8aa-5d79-4955-b1ec-bdaf7f1b6431) service to localhost/127.0.0.1:55924 34246 [refreshUsed-/private/var/folders/98/mxq3vc_n6l5728rf1wmcwrqs52lpwg/T/temp1791820148926982977/dfs/data/data1/current/BP-1058834949-10.0.0.2-1597189606506] WARN org.apache.hadoop.fs.CachingGetSpaceUsed - Thread Interrupted waiting to refresh disk information: sleep interrupted 34247 [refreshUsed-/private/var/folders/98/mxq3vc_n6l5728rf1wmcwrqs52lpwg/T/temp1791820148926982977/dfs/data/data2/current/BP-1058834949-10.0.0.2-1597189606506] WARN org.apache.hadoop.fs.CachingGetSpaceUsed - Thread Interrupted waiting to refresh disk information: sleep interrupted 37192 [HBase-Metrics2-1] WARN org.apache.hadoop.metrics2.impl.MetricsConfig - Cannot locate configuration: tried hadoop-metrics2-datanode.properties,hadoop-metrics2.properties 43904 [master/iad1-ws-cor-r12:0:becomeActiveMaster-SendThread(localhost:58768)] WARN org.apache.zookeeper.ClientCnxn - Session 0x173dfeb0c8b0004 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) [INFO] [INFO] Results: [INFO] [ERROR] Errors: [ERROR] org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V [ERROR] org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V [ERROR] org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V [ERROR] org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V [ERROR] org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V [ERROR] org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V [ERROR] org.eclipse.jetty.server.session.SessionHandler.setHttpOnly(Z)V [INFO] [ERROR] Tests run: 10, Failures: 0, Errors: 7, Skipped: 0 [INFO]
Basically currently Hudi and it's dependency Javalin depend on Jetty 9.4.x but Hbase depends on jetty 9.3.x. And they have incompatible APIs which could not be easily resolved.
Attachments
Attachments
Issue Links
- blocks
-
HUDI-2955 Upgrade Hadoop to 3.3.x
- Patch Available
-
HUDI-3097 Address dependency issue with hudi-trino-bundle in connector
- Closed
- is related to
-
HUDI-3605 hbase dependency need relocation or use other version for hive3
- Closed
- relates to
-
HUDI-3462 List of fixes to Metadata table after 0.10.1
- Closed
- links to