HBase
  1. HBase
  2. HBASE-2105

All appends are serialized in HLog

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 0.20.2
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      duiod on IRC was asking about locks on the write path so I reviewed it a bit and saw that we are very good at not locking except on row level in HRegion and others but in HLog we actually synchronize everything on updateLock. I'm pretty sure that's not what we want. I guess a RWLock would be more useful.

        Activity

        Hide
        Andrew Purtell added a comment -

        Block branch too.

        Show
        Andrew Purtell added a comment - Block branch too.
        Hide
        stack added a comment -

        I don't think there'll ever be a 0.20.4. Our focus has shifted to 0.21 hbase. I'd suggest that we get this into a 0.20.3 RC2. If it adds some speedup to server, it'd be a nice addition.

        Show
        stack added a comment - I don't think there'll ever be a 0.20.4. Our focus has shifted to 0.21 hbase. I'd suggest that we get this into a 0.20.3 RC2. If it adds some speedup to server, it'd be a nice addition.
        Hide
        ryan rawson added a comment -

        I think the underlying sequence file is synchronized, so I'm not sure how much this would help...

        Show
        ryan rawson added a comment - I think the underlying sequence file is synchronized, so I'm not sure how much this would help...
        Hide
        stack added a comment -

        Yeah, I was going to say what Ryan says, that down in SequenceFile, its synchronized anyway so we'd be narrowing the synchronization for sure but ain't sure the narrowing would make for that much in performance improvement.

        Chatting too w/ J-D, we probably want the current synchronization to ensure that the get of the next sequence number happens in the same synchronization block that does the actual write. Otherwise, we could have edits going in out of order which could be a prob. if a high number edit goes in before a lower number edit.... we might lose a few edits if we crash after the high number goes in but before the lower edits have a chance to go in.

        We might want to revisit if we want to run a pool of WAL-writers.

        Show
        stack added a comment - Yeah, I was going to say what Ryan says, that down in SequenceFile, its synchronized anyway so we'd be narrowing the synchronization for sure but ain't sure the narrowing would make for that much in performance improvement. Chatting too w/ J-D, we probably want the current synchronization to ensure that the get of the next sequence number happens in the same synchronization block that does the actual write. Otherwise, we could have edits going in out of order which could be a prob. if a high number edit goes in before a lower number edit.... we might lose a few edits if we crash after the high number goes in but before the lower edits have a chance to go in. We might want to revisit if we want to run a pool of WAL-writers.
        Hide
        stack added a comment -

        This is not a blocker, after discussion. Nor is it to be addressed in 0.20.4. Moving it out.

        Show
        stack added a comment - This is not a blocker, after discussion. Nor is it to be addressed in 0.20.4. Moving it out.
        Hide
        Jean-Daniel Cryans added a comment -

        This is fixed by HDFS-895 and we complement this with HBASE-2467.

        Show
        Jean-Daniel Cryans added a comment - This is fixed by HDFS-895 and we complement this with HBASE-2467 .

          People

          • Assignee:
            Jean-Daniel Cryans
            Reporter:
            Jean-Daniel Cryans
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development