Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-1779

Add support for --hive-database when importing Parquet files into Hive

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4.5
    • Fix Version/s: 1.4.6
    • Component/s: hive-integration
    • Labels:
      None

      Description

      When importing Parquet-formatted data files into Hive, Sqoop leaves the creation of the tables in the metastore to Kite. Unfortunately, the format of the Kite dataset URI for Hive doesn't include the value of the --hive-database option and it only knows how to set the table name from the --hive-table-name option, instead of just using the name of the table being imported.

      As an additional note, running the Parquet importer when HIVE_HOME isn't set causes the import job to fail in a way that isn't very helpful in diagnosing the underlying issue (i.e., that Kite can't find the Hive jars on the classpath.) It would be worthwhile to add better logging to that effect, although I'm not sure if that should be done in Kite or in Sqoop.

        Attachments

        1. SQOOP-parq-hive.patch
          2 kB
          Josh Wills

          Issue Links

            Activity

              People

              • Assignee:
                jwills Josh Wills
                Reporter:
                jwills Josh Wills
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: