Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-2983

OraOop export has degraded performance with wide tables

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 1.4.7
    • None
    • None

    Description

      The current version of OraOOP seems to perform very low from performance POV when --direct mode turned on (regardless if the partitioned feature is turned of).

      Just as a baseline from the current trunk version:
      Inserting 100.000 rows into a 800 column wide Oracle table has 400-600 kb/sec with direct mode on my cluster, while the standard oracle driver can produce up to 1.2-1.8 mb/sec. (depending on the number of mappers, batch size).
      Inserting 1.000.000 rows into the same table goes up to 800k-1mb/sec with OraOOP, however with the standard Oracle connector it's around 3.5mb/sec.

      It seems OraOOP export needs a thorough review and some fixing.

      Attachments

        1. SQOOP-2983-5.patch
          22 kB
          Attila Szabo
        2. SQOOP-2983-6.patch
          24 kB
          Attila Szabo
        3. SQOOP-2983-7.patch
          16 kB
          Attila Szabo

        Issue Links

          Activity

            People

              maugli Attila Szabo
              maugli Attila Szabo
              Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: