Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-685

Sqoop will fail with OutOfMemory on large tables using mysql

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The default MySQL JDBC client behavior is to buffer the entire ResultSet in the client before allowing the user to use the ResultSet object. On large SELECTs, this can cause OutOfMemory exceptions, even when the client intends to close the ResultSet after reading only a few rows. The MySQL ConnManager should configure its connection to use row-at-a-time delivery of results to the client.

        Attachments

        1. MAPREDUCE-685.patch
          15 kB
          Aaron Kimball
        2. MAPREDUCE-685.patch.2
          17 kB
          Aaron Kimball
        3. MAPREDUCE-685.3.patch
          17 kB
          Aaron Kimball

          Activity

            People

            • Assignee:
              kimballa Aaron Kimball
              Reporter:
              kimballa Aaron Kimball
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: