Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-1473

Committed proposal log retains triple the memory it needs to

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 3.5.4, 3.6.0
    • Component/s: server
    • Labels:
      None

      Description

      ZKDatabase.committedLog retains the past 500 transactions to enable fast catch-up. This works great, but it's using triple the memory it needs to by retaining three copies of the data part of any transaction.

      • The first is in committedLog[i].request.request.hb - a heap-allocated ByteBuffer.
      • The second is in committedLog[i].request.txn.data - a jute-serialised record of the transaction
      • The third is in committedLog[i].packet.data - also jute-serialised, seemingly uninitialised data.

      This means that a ZK-server could be using 1G of memory more than it should be in the worst case. We should use just one copy of the data, even if we really have to refer to it 3 times.

        Attachments

        1. ZOOKEEPER-1473.patch
          2 kB
          Thawan Kooburat
        2. ZOOKEEPER-1473.patch
          2 kB
          Thawan Kooburat
        3. ZOOKEEPER-1473.patch
          2 kB
          Michi Mutsuzaki

          Activity

            People

            • Assignee:
              thawan Thawan Kooburat
              Reporter:
              henryr Henry Robinson
            • Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated: