Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-1168 Sqoop2: Delta Fetch/ Merge ( formerly called Incremental Import )
  3. SQOOP-1799

Connector API : Ability for connector to indicate if its FROM and TO support incremental reading/ writing

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • None
    • 1.99.5
    • None
    • None

    Description

      No longer a necessity, If the connectors have delta read/ write configs we will display it and they will use those config values to do the appropriate form or reading from and writing to the data source. At this point having this in the initializer API does not seem necessary, we can revisit if we need this information upfront for any form of validation when the job is created.

      By default it is assumed the connectors will do a full fetch and full write from clean slate.
      For instance if the TO does not support delta records to be written is some fashion, but the FROM side only gave subset of records, we cannot expect delta append or merge ( overwriting existing records with no dupes) to happen.

      Attachments

        Activity

          People

            vybs Veena Basavaraj
            vybs Veena Basavaraj
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: