Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-1168 Sqoop2: Delta Fetch/ Merge ( formerly called Incremental Import )
  3. SQOOP-1799

Connector API : Ability for connector to indicate if its FROM and TO support incremental reading/ writing

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: 1.99.5
    • Component/s: None
    • Labels:
      None

      Description

      No longer a necessity, If the connectors have delta read/ write configs we will display it and they will use those config values to do the appropriate form or reading from and writing to the data source. At this point having this in the initializer API does not seem necessary, we can revisit if we need this information upfront for any form of validation when the job is created.

      By default it is assumed the connectors will do a full fetch and full write from clean slate.
      For instance if the TO does not support delta records to be written is some fashion, but the FROM side only gave subset of records, we cannot expect delta append or merge ( overwriting existing records with no dupes) to happen.

        Attachments

          Activity

            People

            • Assignee:
              vybs Veena Basavaraj
              Reporter:
              vybs Veena Basavaraj

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment