Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-812

JDBC optimizations: setReadOnly, setMaxRows

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.3
    • 1.4
    • None

    Description

      I'm looking at the DataImport code as of Solr v1.3 and using it with Postgres and very large data sets and there some improvement suggestions I have.

      1. call setReadOnly(true) on the connection. DIH doesn't change the data so this is obvious.

      2. call setAutoCommit(false) on the connection. (this is needed by Postgres to ensure that the fetchSize hint actually works)

      3. call setMaxRows(X) on the statement which is to be used when the dataimport.jsp debugger is only grabbing X rows. fetchSize is just a hint and alone it isn't sufficient.

      Attachments

        1. SOLR-812.patch
          8 kB
          Shalin Shekhar Mangar
        2. SOLR-812.patch
          8 kB
          Shalin Shekhar Mangar

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            shalin Shalin Shekhar Mangar
            dsmiley David Smiley
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment