[MAPREDUCE-7368] DBOutputFormat.DBRecordWriter#write must throw exception when it fails

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.1
• Fix Version/s: 3.4.0
• Component/s: mrv2
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.3.1
• Hadoop Flags: Reviewed
• Assignee: Stamatis Zampetakis
• Reporter: Stamatis Zampetakis
• Created: 17/Nov/21 16:01
• Updated: 11/Feb/24 23:19
• Resolved: 08/Dec/21 11:11

Description

When the DBRecordWriter#write fails with an SQLException the problem is not propagated but printed in System.err instead.

public void write(K key, V value) throws IOException {
      try {
        key.write(statement);
        statement.addBatch();
      } catch (SQLException e) {
        e.printStackTrace();
      }
    }

The consumer of this API has no way to tell that the write failed. Moreover, the exception is not present in the logs which makes the problem very hard debug and can easily lead to data corruption since clients can easily assume that everything went well.

Issue Links

• relates to: HIVE-25717 INSERT INTO on external MariaDB/MySQL table fails silently
• supercedes: MAPREDUCE-2548 Log improvements in DBOutputFormat.java and CounterGroup.java
• links to: GitHub Pull Request #3671