[OOZIE-3089] Oozie Sqoop Action Hive Import does not show hive.log contents

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: action
• Assignee: Unassigned
• Reporter: Prabhu Joseph
• Created: 17/Oct/17 08:36
• Updated: 17/Oct/17 08:36

Description

Oozie Sqoop Action which does Hive Import does not show the contents of hive.log in Application Logs. When the Sqoop action fails at HiveImport, the application logs will show only the exit code

Intercepting System.exit(1)
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SqoopMain], exit code [1]

but the actual error message will be written in hive.log under MapTask container directory. This file will be removed by YARN and difficult to debug the issue with just application logs later.

/hadoop/yarn/local/usercache/hive/appcache/application_1507806227070_0028/container_e20_1507806227070_0028_01_000002/tmp/yarn/hive.log

2017-10-17 07:48:21,397 ERROR [main]: tool.ImportTool (ImportTool.java:run(623)) - Encountered IOException running import job: java.io.IOException: Exception thrown in Hive
        at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:363)
        at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:246)
        at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:524)
        at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:615)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:225)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:243)
        at org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:197)
        at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:179)
        at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:58)
        at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:48)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:237)
        at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
        at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:170)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
        at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:164)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:336)
        ... 26 more
Caused by: java.lang.RuntimeException: org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of tez jars, tez.lib.uris is not defined in the configuration
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:560)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
        ... 31 more
Caused by: org.apache.tez.dag.api.TezUncheckedException: Invalid configuration of tez jars, tez.lib.uris is not defined in the configuration
        at org.apache.tez.client.TezClientUtils.setupTezJarsLocalResources(TezClientUtils.java:166)
        at org.apache.tez.client.TezClient.getTezJarResources(TezClient.java:831)
        at org.apache.tez.client.TezClient.start(TezClient.java:355)

Oozie Sqoop Action has to display the contents of hive.log into the application logs.