Uploaded image for project: 'Sqoop'
  1. Sqoop
  2. SQOOP-2983

OraOop export has degraded performance with wide tables

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.4.7
    • Component/s: None
    • Labels:
      None

      Description

      The current version of OraOOP seems to perform very low from performance POV when --direct mode turned on (regardless if the partitioned feature is turned of).

      Just as a baseline from the current trunk version:
      Inserting 100.000 rows into a 800 column wide Oracle table has 400-600 kb/sec with direct mode on my cluster, while the standard oracle driver can produce up to 1.2-1.8 mb/sec. (depending on the number of mappers, batch size).
      Inserting 1.000.000 rows into the same table goes up to 800k-1mb/sec with OraOOP, however with the standard Oracle connector it's around 3.5mb/sec.

      It seems OraOOP export needs a thorough review and some fixing.

        Attachments

        1. SQOOP-2983-5.patch
          22 kB
          Attila Szabo
        2. SQOOP-2983-6.patch
          24 kB
          Attila Szabo
        3. SQOOP-2983-7.patch
          16 kB
          Attila Szabo

          Issue Links

            Activity

              People

              • Assignee:
                maugli Attila Szabo
                Reporter:
                maugli Attila Szabo
              • Votes:
                1 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: