[HIVE-17215] Streaming Ingest API writing unbucketed tables

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.0.0
• Component/s: Transactions
• Target Version/s: 3.0.0
• Assignee: Eugene Koifman
• Reporter: Eugene Koifman
• Created: 31/Jul/17 17:47
• Updated: 22/May/18 23:59
• Resolved: 24/Aug/17 23:19

Description

Currently the API expects the target table to be bucketed.
It creates 1 writer per bucket per connection/partition.
The simplest is to allow the API to create a single writer for unbucketed tables.
If this doesn't provide enough write throughput, the client can create another connection.

Could add a parameter to the API to specify writer parallelism for unbucketed tables. If it's set to 2 for example, the writer will write delta_x_y_0000 and delta_x_y_00001 using statementId. Maybe as a followup.