Uploaded image for project: 'Atlas'
  1. Atlas
  2. ATLAS-2462

Sqoop import for all tables throws NPE for no table provided in command

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 1.0.0-alpha, 0.8.2
    • 1.0.0, 0.8.3
    • atlas-core
    • None

    Description

      Created a cluster and tried importing tables from mysql using sqoop commands.

      • When tried importing a specific table with below command there is no error:
        sqoop import --connect jdbc:mysql://localhost/test --table testtable221 --hive-import --hive-database sqoopimport023 --hive-table hivetesttable221 --username root -P -m 1 --fetch-size 1
      • When tried to import all tables from a specific database using below command, got below error:
        sqoop import-all-tables --connect jdbc:mysql://localhost/test --hive-import --hive-database sqoopimport022 --username root -P -m 1 --fetch-size 1
      17/12/28 05:38:31 WARN mapreduce.PublishJobData: Unable to publish import data to publisher SqoopHook.publish() failed.
      org.apache.atlas.hook.AtlasHookException: SqoopHook.publish() failed.
      	at org.apache.atlas.sqoop.hook.SqoopHook.publish(SqoopHook.java:193)
      	at org.apache.atlas.sqoop.hook.SqoopHook.publish(SqoopHook.java:51)
      	at org.apache.sqoop.mapreduce.PublishJobData.publishJobData(PublishJobData.java:52)
      	at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:284)
      	at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:692)
      	at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:127)
      	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:507)
      	at org.apache.sqoop.tool.ImportAllTablesTool.run(ImportAllTablesTool.java:111)
      	at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
      	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
      	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
      	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:225)
      	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
      	at org.apache.sqoop.Sqoop.main(Sqoop.java:243)
      Caused by: java.lang.NullPointerException
      	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getTableQualifiedName(HiveMetaStoreBridge.java:383)
      	at org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getTableQualifiedName(HiveMetaStoreBridge.java:412)
      	at org.apache.atlas.sqoop.hook.SqoopHook.createHiveTableInstance(SqoopHook.java:88)
      	at org.apache.atlas.sqoop.hook.SqoopHook.publish(SqoopHook.java:183)
      	... 13 more

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ruchi_solani Ruchi Solani
            ruchi_solani Ruchi Solani
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment