Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-22501

Stats reported multiple times during MR execution for UNION queries

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None

    Description

      Take the following example:

      set hive.execution.engine=mr;
      
      create table tb(id string) stored as orc;
      insert into tb values('1');
      create table tb2 like tb stored as orc;
      
      insert into tb2 select * from tb union all select * from tb;
      

      Last insert results in 2 records in the table, but TOTAL_TABLE_ROWS_WRITTEN statistic (and number of affected rows on the consolse) is 4.

      We seem to traverse the operator graph multiple times starting from every TS operator and increment the counters every time we hit the FS operator. UNION-ing the table 3 times results in 9 TOTAL_TABLE_ROWS_WRITTEN.

      Attachments

        1. HIVE-22501.1.patch
          2 kB
          Daniel Voros

        Activity

          People

            dvoros Daniel Voros
            dvoros Daniel Voros
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: