Derby
  1. Derby
  2. DERBY-6096

OutOfMemoryError with Clob or Blob hash join: DataTypeDescriptor.estimatedMemoryUsage() has no case for BLOB or CLOB so would underestimate memory usage for those types at zero

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.1.3.1, 10.2.2.0, 10.3.3.0, 10.4.2.0, 10.5.3.0, 10.6.2.1, 10.7.1.1, 10.8.3.0, 10.9.1.0, 10.10.1.1
    • Fix Version/s: 10.11.1.1
    • Component/s: SQL

      Description

      In discussion on derby-dev regarding how much memory is used for hash joins, Knut noted:

      I haven't verified, but I think HashJoinStrategy uses
      DataTypeDescriptor.estimatedMemoryUsage() to estimate how much memory
      the hash table will consume. That method has no case for BLOB or CLOB,
      so it looks as if it will return zero for LOB columns. If that's so, it
      will definitely overestimate how many rows fits in maxMemoryPerTable
      kilobytes if the rows contain LOBs.

      DataTypeDescriptor.estimatedMemoryUsage() should be updated to include BLOB and CLOB and we should try verify if this theory is correct with a reproduction.

      1. D6096.java
        0.8 kB
        Knut Anders Hatlen
      2. derby-6096_code_diff.txt
        0.9 kB
        Kathey Marsden
      3. derby-6096_diff.txt
        6 kB
        Kathey Marsden
      4. derby-6096_diff2.txt
        6 kB
        Kathey Marsden
      5. less-gc.diff
        2 kB
        Knut Anders Hatlen
      6. releaseNote.html
        2 kB
        Rick Hillegas
      7. releaseNote.html
        4 kB
        Kathey Marsden

        Issue Links

          Activity

          Kathey Marsden created issue -
          Knut Anders Hatlen made changes -
          Field Original Value New Value
          Attachment D6096.java [ 12572104 ]
          Kathey Marsden made changes -
          Summary DataTypeDescriptor.estimatedMemoryUsage() has no case for BLOB or CLOB so would underestimate memory usage for those types at zero OutOfMemoryError with Clob or Blob hash join:DataTypeDescriptor.estimatedMemoryUsage() has no case for BLOB or CLOB so would underestimate memory usage for those types at zero
          Kathey Marsden made changes -
          Summary OutOfMemoryError with Clob or Blob hash join:DataTypeDescriptor.estimatedMemoryUsage() has no case for BLOB or CLOB so would underestimate memory usage for those types at zero OutOfMemoryError with Clob or Blob hash join: DataTypeDescriptor.estimatedMemoryUsage() has no case for BLOB or CLOB so would underestimate memory usage for those types at zero
          Kathey Marsden made changes -
          Attachment derby-6096_code_diff.txt [ 12576571 ]
          Kathey Marsden made changes -
          Attachment derby-6096_diff.txt [ 12576595 ]
          Kathey Marsden made changes -
          Attachment releaseNote.html [ 12576639 ]
          Kathey Marsden made changes -
          Attachment derby-6096_diff2.txt [ 12576646 ]
          Kathey Marsden made changes -
          Assignee Kathey Marsden [ kmarsden ]
          Knut Anders Hatlen made changes -
          Attachment less-gc.diff [ 12576966 ]
          Kathey Marsden made changes -
          Labels derby_backport_reject_10_10
          Kathey Marsden made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Kathey Marsden made changes -
          Fix Version/s 10.11.0.0 [ 12324243 ]
          Gavin made changes -
          Workflow jira [ 12769530 ] Default workflow, editable Closed status [ 12802291 ]
          Knut Anders Hatlen made changes -
          Link This issue is related to DERBY-4620 [ DERBY-4620 ]
          Rick Hillegas made changes -
          Attachment releaseNote.html [ 12649320 ]

            People

            • Assignee:
              Kathey Marsden
              Reporter:
              Kathey Marsden
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development