[ATLAS-3140] Hive Hook throws InvalidTableException when getting metadata of _dummy_database._dummy_table

• Type: Improvement
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 1.0.0, 1.1.0
• Fix Version/s: 1.0.0, 1.1.0
• Component/s: atlas-core
• Assignee: Unassigned
• Reporter: Jiaqi Shan
• Created: 15/Apr/19 07:54
• Updated: 15/Apr/19 08:13

Description

When we run sql like 'select 1+1', following error is logged in hive.log:

2019-04-15 14:47:55,844 ERROR [main]: hook.HiveHook (HiveHook.java:run(167)) - HiveHook.run(): failed to process operation QUERY
org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table not found _dummy_table
        at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1189)
        at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1159)
        at org.apache.atlas.hive.hook.events.BaseHiveEvent.toAtlasEntity(BaseHiveEvent.java:252)
        at org.apache.atlas.hive.hook.events.BaseHiveEvent.getInputOutputEntity(BaseHiveEvent.java:231)
        at org.apache.atlas.hive.hook.events.CreateHiveProcess.getEntities(CreateHiveProcess.java:86)
        at org.apache.atlas.hive.hook.events.CreateHiveProcess.getNotificationMessages(CreateHiveProcess.java:58)
        at org.apache.atlas.hive.hook.HiveHook.run(HiveHook.java:164)
        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1804)
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1424)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1208)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1198)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:223)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:175)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:412)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:804)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:722)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:657)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

The exception is caused by getting metadata from a dummy table. So we use Entity.isDummy() to judge whether the entity is dummy when Hive Hook calling BaseHiveEvent.toAtlasEntity()