[KYLIN-4345] Build Global Dict by MR/Hive, Parallel Part Build Step implementation

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Sprint: Sprint 50
• Assignee: wangxiaojing
• Reporter: wangxiaojing
• Created: 16/Jan/20 02:05
• Updated: 30/May/20 10:30
• Resolved: 30/May/20 10:30

Description

This step mainly implements temporary dictionary coding for each global dict column in parallel through MR and stored in HDFS.

• Example stored in HDFS:
  /user/kylin/globaldic_test/kylin-188c9f9d_dabb_944e_9f20_99dc95be66e6/kylin_sales_cube_mr/dict_column=KYLIN_SALES_SELLER_ID/part_sort
• Content is : temporary dict value and Original value, example like below:
  1 200002222
  2 100001
  3 300001

Issue Links

• links to: GitHub Pull Request #1091