Bookkeeper
  1. Bookkeeper
  2. BOOKKEEPER-196

Define interface between bookie and ledger storage

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.1.0
    • Component/s: None
    • Labels:
      None

      Description

      EntryLogger and LedgerCache are both part of a very interdependent storage mechanism where entries are interleaved in a single log(EntryLogger) and index files are maintained (LedgerCache). I'd like to experiment with some other schemes (Im not convinced the interleaving is required for high performance). ZOOKEEPER-507 brought in these changes, but it also brought in a lot of other stuff, and I think its the other stuff (specifically taking the writing to separate files out of the critical path) which gave us the performance boost. To do this cleanly, we need a well defined storage interface for this. This JIRA is to provide this. Future work can move the interleaved implementation into another package as org.apache.bookkeeper.bookie is getting a little crowded now.

      1. BOOKKEEPER-196.diff
        19 kB
        Ivan Kelly
      2. BOOKKEEPER-196.diff
        28 kB
        Ivan Kelly
      3. BOOKKEEPER-196.diff
        27 kB
        Ivan Kelly

        Activity

        Hide
        Sijie Guo added a comment -

        the patch is good to me, although seems that the patch should be updated to latest trunk after BOOKKEEPER-135 is in.

        Show
        Sijie Guo added a comment - the patch is good to me, although seems that the patch should be updated to latest trunk after BOOKKEEPER-135 is in.
        Hide
        Ivan Kelly added a comment -

        Rebased onto trunk

        Show
        Ivan Kelly added a comment - Rebased onto trunk
        Hide
        Sijie Guo added a comment -

        the new patch is OK to me. +1

        Show
        Sijie Guo added a comment - the new patch is OK to me. +1
        Hide
        Ivan Kelly added a comment -

        Committed as r1327027. Thanks for reviewing Sijie.

        Show
        Ivan Kelly added a comment - Committed as r1327027. Thanks for reviewing Sijie.
        Hide
        Hudson added a comment -

        Integrated in bookkeeper-trunk #462 (See https://builds.apache.org/job/bookkeeper-trunk/462/)
        BOOKKEEPER-196: Define interface between bookie and ledger storage (ivank) (Revision 1327027)

        Result = ABORTED
        ivank :
        Files :

        • /zookeeper/bookkeeper/trunk/CHANGES.txt
        • /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java
        • /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java
        • /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java
        • /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/HandleFactoryImpl.java
        • /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/InterleavedLedgerStorage.java
        • /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDescriptor.java
        • /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDescriptorImpl.java
        • /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDescriptorReadOnlyImpl.java
        • /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerStorage.java
        Show
        Hudson added a comment - Integrated in bookkeeper-trunk #462 (See https://builds.apache.org/job/bookkeeper-trunk/462/ ) BOOKKEEPER-196 : Define interface between bookie and ledger storage (ivank) (Revision 1327027) Result = ABORTED ivank : Files : /zookeeper/bookkeeper/trunk/CHANGES.txt /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Bookie.java /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/EntryLogger.java /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/GarbageCollectorThread.java /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/HandleFactoryImpl.java /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/InterleavedLedgerStorage.java /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDescriptor.java /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDescriptorImpl.java /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerDescriptorReadOnlyImpl.java /zookeeper/bookkeeper/trunk/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/LedgerStorage.java

          People

          • Assignee:
            Ivan Kelly
            Reporter:
            Ivan Kelly
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development