Uploaded image for project: 'Apache AsterixDB'
  1. Apache AsterixDB
  2. ASTERIXDB-2656

No need for frame constraint in probe phase of hybrid hash join

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: *DB - AsterixDB
    • Labels:
      None

      Description

      In an optimized hybrid hash join, there is a constraint that a spilled partition cannot have more than 1 frame. While it seems to be necessary during the build phase to help with having as much data for in-memory partitions, it seems an unnecessary constraint during the probe phase.

      During the probe, we should let the spilled partitions to grow and use the leftover frames not used by in-memory partitions. This also solves a bug that hybrid hash join currently has.

      During the probe phase, in case there is not enough memory for inserting a large record, asterixdb may either flush the record directly to the disk or spill the largest spilled partition to the disk, depending on which one is larger. In the case that the biggest spilled partition is the vicitim to flush and release the frames, the large record cannot use the freed frames as it belongs to a spilled partition(otherwise it would have been probed) and as such 1-frame limit is applied on it.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              shivaj Shiva Jahangiri
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: