Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-19671

Distribute by rand() can lead to data inconsistency

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Noticed the following queries can give different results:

      select count(*) from tbl;
select count(*) from (select * from tbl distribute by rand()) a;

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. KYLIN-3388 Data may become not correct if mappers fail during the redistribute step, "distribute by rand()"

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. SPARK-24607 Distribute by rand() can lead to data inconsistency

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HIVE-13108 Operators: SORT BY randomness is not safe with network partitions

          • Major - Major loss of function.
          • Closed

          Bug - A problem which impairs or prevents the functions of the product. HIVE-20220 Incorrect result when hive.groupby.skewindata is enabled

          • Major - Major loss of function.
          • Patch Available

          Activity

            People

              lirui Rui Li
              lirui Rui Li
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated: