Uploaded image for project: 'Apache AsterixDB'
  1. Apache AsterixDB
  2. ASTERIXDB-2577

Not keeping one frame for each spilled partition before starting the probe phase in hybrid hash join

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.9.4.1
    • Fix Version/s: None
    • Component/s: RT - Runtime
    • Labels:
      None

      Description

      In probe() method in optimized hybrid hash join, if insertion fails on the current spilled partition, we try to find the biggest spilled partition and flush it as a victim. If we could not find any spilled partition with size > 0, then we ASSUME that the record is large and flush it as a big object. By running customerOrderCIDHybridHashJoin_Case3() test in 

      TPCHCustomerOrderHashJoinTest, it can be seen that the record size is 206 bytes (so it is smaller than a frame), but neither the spilled partitions nor the buffer manager has any frame (This is the problem, there should be 1 frame for each spilled partition). In this case, we flush the record as a large object. This means that every single record that is supposed to get inserted to a spilled partition during the probe, will get flushed separately. 

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              shivaj Shiva Jahangiri
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: