Sqoop
  1. Sqoop
  2. SQOOP-863

Sqoop2: Introduce ProgressThread into Extractor and Loader

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.99.1
    • Fix Version/s: 1.99.2
    • Component/s: None
    • Labels:
      None

      Description

      We need to introduce extractor and loader ProgressThread so that we can report progress. This is very critical for long running tasks as we need to propagate the progress up to the mapreduce engine so that our tasks won't get killed.

      1. bugSQOOP-863.patch
        6 kB
        Jarek Jarcec Cecho
      2. bugSQOOP-863.patch
        6 kB
        Jarek Jarcec Cecho
      3. bugSQOOP-863.patch
        9 kB
        Jarek Jarcec Cecho

        Issue Links

          Activity

          Jarek Jarcec Cecho created issue -
          Jarek Jarcec Cecho made changes -
          Field Original Value New Value
          Link This issue is blocked by SQOOP-864 [ SQOOP-864 ]
          Jarek Jarcec Cecho made changes -
          Link This issue duplicates SQOOP-752 [ SQOOP-752 ]
          Jarek Jarcec Cecho made changes -
          Summary Sqoop2: Introduce Reporter into Extractor and Loader Sqoop2: Introduce ProgressThread into Extractor and Loader
          Description We need to introduce extractor and loader reporter objects so that we can provide the underlying objects ability to report progress. This is very critical for long running tasks as we need to propagate the progress up to the mapreduce engine so that our tasks won't get killed. We need to introduce extractor and loader {{ProgressThread}} so that we can report progress. This is very critical for long running tasks as we need to propagate the progress up to the mapreduce engine so that our tasks won't get killed.
          Jarek Jarcec Cecho made changes -
          Remote Link This issue links to "Review board (Web Link)" [ 12069 ]
          Jarek Jarcec Cecho made changes -
          Attachment bugSQOOP-863.patch [ 12573011 ]
          Jarek Jarcec Cecho made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Jarek Jarcec Cecho made changes -
          Remote Link This issue links to "Review board (Web Link)" [ 12100 ]
          Jarek Jarcec Cecho made changes -
          Remote Link This issue links to "Review board (Web Link)" [ 12100 ]
          Jarek Jarcec Cecho made changes -
          Attachment bugSQOOP-863.patch [ 12575252 ]
          Jarek Jarcec Cecho made changes -
          Attachment bugSQOOP-863.patch [ 12575624 ]
          Kathleen Ting made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]

            People

            • Assignee:
              Jarek Jarcec Cecho
              Reporter:
              Jarek Jarcec Cecho
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development