Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-3479

Incremental import on existing table in Hive should throw exception if wrong file format is used

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.4.7
    • Fix Version/s: None
    • Component/s: sqoop2-client
    • Labels:
      None

      Description

      Performing an incremental import on an existing table in Hive will corrupt that table if the import is performed with a different file format (for example textfile) than existing files in the table (for example parquet). Rather than corrupting the table, sqoop could check the file format and throw an error if the formats aren't the same.

       

      This happens when you manually provide flags

      --incremental append --check-column [somecolumn] --last-value [somevalue]

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              wessel.valkenburg.tdh Wessel Valkenburg
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: