Pig
  1. Pig
  2. PIG-3597

Enable piggybank DBStorage to store to JDBC databases without Batch Statement support

    Details

      Description

      There are no option to turning batching off with this storage.

      Some JDBC driver doesn't provide Batch Statement implementation.

      org.apache.pig.piggybank.storage.DBStorage.java:157
            this.ps.addBatch();
            this.count += 1;
            if (this.count > this.batchSize) {
              this.count = 0;
              this.ps.executeBatch();
              this.ps.clearBatch();
              this.ps.clearParameters();
            }
      

      An easy fix is to use DatabaseMetaData.supportsBatchUpdates() in JDBC.

      And to use only ps.execute() if the JDBC driver doesn't support Batch Statement.

        Issue Links

          Activity

          Nezih Yigitbasi made changes -
          Link This issue is related to PIG-3770 [ PIG-3770 ]
          Damien Carol made changes -
          Description There are no option to turning batching off with this storage.

          Some exotic JDBC driver doesn't provide Batch Statement implementation.

          {code:title=org.apache.pig.piggybank.storage.DBStorage.java:157|borderStyle=solid}
                this.ps.addBatch();
                this.count += 1;
                if (this.count > this.batchSize) {
                  this.count = 0;
                  this.ps.executeBatch();
                  this.ps.clearBatch();
                  this.ps.clearParameters();
                }
          {code}

          An easy fix is to use DatabaseMetaData.supportsBatchUpdates() in JDBC.

          And to use only ps.execute() if the JDBC driver doesn't support Batch Statement.
          There are no option to turning batching off with this storage.

          Some JDBC driver doesn't provide Batch Statement implementation.

          {code:title=org.apache.pig.piggybank.storage.DBStorage.java:157|borderStyle=solid}
                this.ps.addBatch();
                this.count += 1;
                if (this.count > this.batchSize) {
                  this.count = 0;
                  this.ps.executeBatch();
                  this.ps.clearBatch();
                  this.ps.clearParameters();
                }
          {code}

          An easy fix is to use DatabaseMetaData.supportsBatchUpdates() in JDBC.

          And to use only ps.execute() if the JDBC driver doesn't support Batch Statement.
          Damien Carol made changes -
          Field Original Value New Value
          Labels io jdbc jdbcstorage pig piggybank
          Tags io jdbc sql pig
          Damien Carol created issue -

            People

            • Assignee:
              Unassigned
              Reporter:
              Damien Carol
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Development