[BEAM-8407] [SQL] Some Hive tests throw NullPointerException, but get marked as passing (Direct Runner)

• Type: Bug
• Status: Open
• Priority: P1
• Resolution: Unresolved
• Affects Version/s: 2.15.0
• Component/s: dsl-sql
• Labels: test
• Assignee: Unassigned
• Reporter: Kirill Kozlov
• Created: 15/Oct/19 23:56
• Updated: 04/Jun/22 14:38

Description

One of the tests with this issue:

./gradlew -p sdks/java/extensions/sql/ test --tests org.apache.beam.sdk.extensions.sql.meta.provider.hcatalog.BeamSqlHiveSchemaTest.testSelectFromImplicitDefaultDb --info

Stack trace:

[direct-runner-worker] ERROR hive.log - Got exception: java.lang.NullPointerException null[direct-runner-worker] ERROR hive.log - Got exception: java.lang.NullPointerException nulljava.lang.NullPointerException 
at org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.getFilteredObjects(AuthorizationMetaStoreFilterHook.java:77) ~[hive-exec-2.1.0.jar:2.1.0] 
at org.apache.hadoop.hive.ql.security.authorization.plugin.AuthorizationMetaStoreFilterHook.filterDatabases(AuthorizationMetaStoreFilterHook.java:54) ~[hive-exec-2.1.0.jar:2.1.0] 
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabases(HiveMetaStoreClient.java:1126) [hive-metastore-2.1.0.jar:2.1.0] 
at org.apache.hive.hcatalog.common.HiveClientCache$CacheableHiveMetaStoreClient.isOpen(HiveClientCache.java:369) [hive-hcatalog-core-2.1.0.jar:2.1.0] 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_222] 
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_222] 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_222] 
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_222] 
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:150) [hive-metastore-2.1.0.jar:2.1.0] 
at com.sun.proxy.$Proxy56.isOpen(Unknown Source) [?:?] 
at org.apache.hive.hcatalog.common.HiveClientCache.get(HiveClientCache.java:205) [hive-hcatalog-core-2.1.0.jar:2.1.0] 
at org.apache.hive.hcatalog.common.HCatUtil.getHiveMetastoreClient(HCatUtil.java:558) [hive-hcatalog-core-2.1.0.jar:2.1.0] 
at org.apache.beam.sdk.io.hcatalog.HCatalogUtils.createMetaStoreClient(HCatalogUtils.java:42) [beam-sdks-java-io-hcatalog-2.17.0-SNAPSHOT.jar:?] 
at org.apache.beam.sdk.io.hcatalog.HCatalogIO$BoundedHCatalogSource.getEstimatedSizeBytes(HCatalogIO.java:323) [beam-sdks-java-io-hcatalog-2.17.0-SNAPSHOT.jar:?] 
at org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.startDynamicSplitThread(BoundedReadEvaluatorFactory.java:172) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
at org.apache.beam.runners.direct.BoundedReadEvaluatorFactory$BoundedReadEvaluator.processElement(BoundedReadEvaluatorFactory.java:148) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
at org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:160) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
at org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:124) [beam-runners-direct-java-2.17.0-SNAPSHOT-unshaded.jar:?] 
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_222] 
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_222] 
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_222] 
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_222] 
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_222]

... more NullPointerExceptions