[PIG-3597] Enable piggybank DBStorage to store to JDBC databases without Batch Statement support

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.12.0
• Component/s: piggybank
• Labels: io jdbc jdbcstorage pig piggybank
• Assignee: Unassigned
• Reporter: Damien Carol
• Created: 28/Nov/13 17:07
• Updated: 18/Feb/14 23:22

Description

There are no option to turning batching off with this storage.

Some JDBC driver doesn't provide Batch Statement implementation.

org.apache.pig.piggybank.storage.DBStorage.java:157

      this.ps.addBatch();
      this.count += 1;
      if (this.count > this.batchSize) {
        this.count = 0;
        this.ps.executeBatch();
        this.ps.clearBatch();
        this.ps.clearParameters();
      }

An easy fix is to use DatabaseMetaData.supportsBatchUpdates() in JDBC.

And to use only ps.execute() if the JDBC driver doesn't support Batch Statement.

Issue Links

• is related to: PIG-3770 Enhance DBStorage to make it more flexible (should batch statements?, rollback on job failure, support command line arguments etc.)