[SENTRY-1898] Sentry no longer supports creating more than ~15 partitions at once

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Component/s: Sentry
• Assignee: Na Li
• Reporter: Na Li
• Created: 23/Aug/17 19:27
• Updated: 24/Aug/17 02:44
• Resolved: 24/Aug/17 02:44

Description

Vamsee reported: I notice this problem on our stress cluster where we are creating a huge amounts of Hive metadata (Databases, Tables, and Partitions). During this process, multiple partitions of the table are being created at one go using:
ALTER TABLE <TB> ADD PARTITION <PART1> ADD PARTITION <PART2>...
I notice that when we add more than 20 partitions here in one sql statement, Sentry fails to process the transaction as we cross the 4000 character threshold on PATH change table.

Every partition entry is of format:
{"1":

{"str":"test_hivemd_db_2017_08_15_08_14_27_2_0_8.test_hivemd_table_2017_08_15_08_14_27_2_0_0"}

,"2":

{"lst":["lst",1,["str",6,"user","hive","warehouse","test_hivemd_db_2017_08_15_08_14_27_2_0_8.db","test_hivemd_table_2017_08_15_08_14_27_2_0_0","part=test_hivemd_partition_2017_08_15_08_14_27_2_0_1"]]}

,"3":{"lst":["lst",0]}}
~ 327 characters
So to fit in 4000 characters, we are limited to 10-15 partitions to be created at one go.

We should allow the big path update from hive.

Issue Links

• links to: Review Link for this case