Details
-
Sub-task
-
Status: Closed
-
Major
-
Resolution: Fixed
-
3.0.0
Description
To support ACID replication, we will be introducing a per table write Id which will replace the transaction id in the primary key for each row in a ACID table.
The current primary key is determined via
<original transaction id, bucketid , rowid>
which will move to
<original write id, bucketid, rowid>
For each table modified by the given transaction will have a table level write ID allocated and a persisted map of global txn id -> to table -> write id for that table has to be maintained to allow Snapshot isolation.
Readers should use the combination of ValidTxnList and ValidWriteIdList(Table) for snapshot isolation.
Hive Replication - ACID Tables.pdf has a section "Per Table Sequences (Write-Id)" with more detials
Attachments
Attachments
Issue Links
- is depended upon by
-
HIVE-18193 Migrate existing ACID tables to use write id per table rather than global transaction id
- Closed
- is related to
-
HIVE-18444 when creating transactional table make sure location has no data
- Open
- relates to
-
HIVE-18748 Rename table impacts the ACID behavior as table names are not updated in meta-tables.
- Closed
-
HIVE-18750 Exchange partition should be disabled on ACID/Insert-only tables with per table write ID.
- Closed
-
HIVE-18751 ACID table scan through get_splits UDF doesn't receive ValidWriteIdList configuration.
- Closed
-
HIVE-18753 Correct methods and variables names which uses writeId instead of transactionId.
- Closed
-
HIVE-18824 ValidWriteIdList config should be defined on tables which has to collect stats after insert
- Closed
-
HIVE-18864 ValidWriteIdList snapshot seems incorrect if obtained after allocating writeId by current transaction.
- Closed
-
HIVE-18747 Cleaner for TXN_TO_WRITE_ID table entries using MIN_HISTORY_LEVEL.
- Closed
-
HIVE-18749 Need to replace transactionId with writeId in RecordIdentifier and other relevant contexts.
- Closed
-
HIVE-18752 HiveEndPoint, HiveStreamingConnection: Optimise metastore calls to open transactions batch and allocate write Ids.
- Open
-
HIVE-18923 ValidWriteIdList snapshot per table can be cached for multi-statement transactions.
- Open
- links to