Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6794

Document the JDBC properties required to retrieve result sets in batches while querying large tables

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.14.0
    • Fix Version/s: None
    • Component/s: Documentation
    • Labels:

      Description

      Document the JDBC properties required to retrieve result sets in batches while querying large tables

      Querying large tables using JDBC plugin causes OOM as most JDBC drivers cache the entire result set at the client by default.

      To avoid this additional parameters needs to be specified with the JDBC connection string so that the driver fetches records in batches and reloads when exhausted.

      For postgres driver set autocommit mode to false - jdbc:postgresql://url:port/schema?defaultAutoCommit=false

      Links

      [1] https://issues.apache.org/jira/browse/DRILL-4177

      [2] https://jdbc.postgresql.org/documentation/93/query.html#fetchsize-example

      [3] https://www.postgresql.org/docs/9.3/static/ecpg-sql-set-autocommit.html

      [4] https://jdbc.postgresql.org/documentation/head/ds-cpds.htm

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                bbevens Bridget Bevens
                Reporter:
                rajrahul Rahul Raj
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: